Jul 07 11:15:17.229472 ip-10-0-133-180 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 11:15:17.650897 ip-10-0-133-180 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 11:15:17.650897 ip-10-0-133-180 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 11:15:17.650897 ip-10-0-133-180 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 11:15:17.650897 ip-10-0-133-180 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 11:15:17.650897 ip-10-0-133-180 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 11:15:17.653030 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.652943    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 11:15:17.662178 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662158    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:15:17.662178 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662178    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662186    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662205    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662210    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662214    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662218    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662223    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662227    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662231    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662235    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662239    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662243    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662246    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662251    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662255    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662260    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662263    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662268    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662271    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662275    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:15:17.662340 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662279    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662283    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662287    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662296    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662300    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662304    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662308    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662313    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662317    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662321    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662325    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662330    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662335    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662339    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662344    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662348    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662352    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662356    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662361    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662365    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:15:17.663137 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662369    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662373    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662377    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662382    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662386    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662390    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662394    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662399    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662403    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662407    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662412    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662416    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662420    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662425    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662429    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662433    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662438    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662442    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662446    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:15:17.663997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662450    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662454    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662459    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662463    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662467    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662472    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662476    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662481    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662485    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662490    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662494    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662498    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662502    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662510    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662517    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662524    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662529    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662533    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662538    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:15:17.664815 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662542    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662547    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662551    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662555    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662561    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662565    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662569    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662574    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662578    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662583    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662588    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662592    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662595    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662599    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662604    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662608    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662612    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662616    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662620    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.662624    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:15:17.665634 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663815    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663824    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663829    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663834    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663839    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663844    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663848    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663852    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663859    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663865    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663870    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663875    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663880    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663884    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663889    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663893    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663898    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663904    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663908    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663912    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:15:17.666490 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663916    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663920    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663924    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663929    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663933    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663937    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663941    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663945    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663949    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663953    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663957    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663961    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663967    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663972    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663976    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663980    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663984    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663988    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663992    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.663996    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:15:17.667239 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664000    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664003    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664007    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664015    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664019    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664024    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664028    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664033    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664038    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664043    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664049    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664053    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664057    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664062    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664066    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664070    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664075    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664079    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664083    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:15:17.667776 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664087    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664092    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664097    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664103    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664107    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664112    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664117    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664121    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664125    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664129    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664135    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664139    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664143    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664148    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664152    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664156    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664160    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664165    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664169    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:15:17.668289 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664173    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664177    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664181    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664185    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664207    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664213    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664219    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664223    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664227    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664235    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664239    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664243    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664248    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664252    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664256    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664260    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664264    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664268    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664273    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664278    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:15:17.668965 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664282    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664882    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664890    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664894    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664898    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664903    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664907    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664911    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664916    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664921    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664925    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664929    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664934    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664938    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664942    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664946    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664950    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664960    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664965    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664970    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664974    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:15:17.669552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664979    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664984    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664989    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664993    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.664999    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665005    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665010    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665014    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665019    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665023    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665028    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665032    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665037    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665041    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665046    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665050    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665054    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665058    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665062    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:15:17.670049 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665067    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665071    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665075    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665079    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665083    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665096    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665100    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665105    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665109    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665113    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665117    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665121    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665125    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665130    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665136    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665141    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665145    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665149    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665153    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:15:17.670624 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665157    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665161    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665166    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665170    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665174    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665179    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665183    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665187    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665207    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665212    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665219    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665223    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665228    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665232    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665236    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665239    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665243    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665248    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665252    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:15:17.671073 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665256    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665260    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665264    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665268    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665273    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665278    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665283    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665288    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665292    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665296    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665301    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665305    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665309    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665313    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665317    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665321    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665325    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665330    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665334    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:15:17.671561 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665339    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665345    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.665351    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665447    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665458    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665472    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665479    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665485    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665493    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665500    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665506    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665511    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665516    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665522    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665527    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665532    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665537    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665542    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665546    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665551    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665557    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665564    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665568    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 11:15:17.672010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665573    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665578    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665584    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665590    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665594    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665599    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665605    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665610    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665614    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665619    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665624    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665629    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665639    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665643    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665648    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665652    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665657    2582 flags.go:64] FLAG: --enable-server="true"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665663    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665672    2582 flags.go:64] FLAG: --event-burst="100"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665677    2582 flags.go:64] FLAG: --event-qps="50"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665682    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665687    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665691    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665698    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 11:15:17.672598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665702    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665707    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665712    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665717    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665721    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665738    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665743    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665747    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665752    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665757    2582 flags.go:64] FLAG: --feature-gates=""
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665772    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665777    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665781    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665786    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665791    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665796    2582 flags.go:64] FLAG: --help="false"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665801    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-133-180.ec2.internal"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665806    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665810    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665815    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665820    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665826    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665831    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 11:15:17.673168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665835    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665839    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665846    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665851    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665856    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665860    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665864    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665869    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665874    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665878    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665883    2582 flags.go:64] FLAG: --lock-file=""
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665887    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665891    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665896    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665906    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665912    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665917    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665921    2582 flags.go:64] FLAG: --logging-format="text"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665926    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665931    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665936    2582 flags.go:64] FLAG: --manifest-url=""
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665941    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665947    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665952    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665959    2582 flags.go:64] FLAG: --max-pods="110"
Jul 07 11:15:17.673871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665963    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665968    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665972    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665977    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665982    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665987    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.665991    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666002    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666007    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666013    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666018    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666023    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666030    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666034    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666040    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666044    2582 flags.go:64] FLAG: --port="10250"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666049    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666053    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-043e0960d239a5abb"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666058    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666063    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666068    2582 flags.go:64] FLAG: --register-node="true"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666072    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666081    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666091    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 07 11:15:17.674494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666095    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666100    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666106    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666110    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666116    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666120    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666125    2582 flags.go:64] FLAG: --runonce="false"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666129    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666134    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666139    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666144    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666148    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666153    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666158    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666162    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666167    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666172    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666178    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666182    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666187    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666211    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666216    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666224    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666229    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666234    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 11:15:17.675072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666240    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666245    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666250    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666254    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666258    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666265    2582 flags.go:64] FLAG: --v="2"
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666271    2582 flags.go:64] FLAG: --version="false"
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666279    2582 flags.go:64] FLAG: --vmodule=""
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666284    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.666290    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666468    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666475    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666480    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666485    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666490    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666494    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666498    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666501    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666506    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666510    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666514    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666519    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:15:17.675687 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666523    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666527    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666533    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666537    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666542    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666546    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666550    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666557    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666562    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666567    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666572    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666576    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666581    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666585    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666589    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666595    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666599    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666604    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666609    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:15:17.676266 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666613    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666617    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666621    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666625    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666630    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666634    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666638    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666641    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666646    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666650    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666656    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666662    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666666    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666672    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666676    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666682    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666687    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666691    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666696    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666700    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:15:17.676720 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666705    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666709    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666713    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666717    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666721    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666726    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666730    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666735    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666741    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666745    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666749    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666754    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666759    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666763    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666767    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666772    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666778    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666782    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666786    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:15:17.677262 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666790    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666794    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666798    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666803    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666807    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666811    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666815    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666819    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666825    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666830    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666834    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666838    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666842    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666846    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666851    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666855    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666860    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666864    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666868    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:15:17.677723 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666872    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666876    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666883    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666888    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666892    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666896    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666901    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666906    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666910    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.666914    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.667641    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.675348    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.675369    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.676274    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.676281    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 11:15:17.678176 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676343    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676348    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676351    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676355    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676358    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676361    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676364    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676367    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676370    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676373    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676375    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676378    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676381    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676383    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676386    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676388    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676391    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676394    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676397    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676399    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:15:17.678572 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676402    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676405    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676407    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676409    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676412    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676414    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676417    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676420    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676422    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676432    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676435    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676438    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676440    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676443    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676445    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676448    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676451    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676455    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676458    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:15:17.679064 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676461    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676463    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676466    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676468    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676471    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676473    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676476    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676478    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676481    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676483    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676485    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676488    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676491    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676493    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676496    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676498    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676501    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676503    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676507    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:15:17.679544 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676510    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676513    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676515    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676518    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676527    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676529    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676532    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676534    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676537    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676539    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676542    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676544    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676547    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676549    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676552    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676555    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676559    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676561    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676564    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676566    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:15:17.679992 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676569    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676571    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676574    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676576    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676579    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676581    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676584    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676587    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676589    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676591    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676594    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676597    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676599    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676602    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676604    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676607    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676609    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676617    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676620    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676623    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:15:17.680487 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676625    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.676630    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676777    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676781    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676784    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676786    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676789    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676792    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676795    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676797    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676800    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676802    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676805    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676807    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676810    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:15:17.680975 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676812    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676815    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676817    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676820    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676822    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676825    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676827    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676830    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676832    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676835    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676837    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676839    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676842    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676845    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676847    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676856    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676859    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676862    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676864    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:15:17.681356 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676867    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676869    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676873    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676876    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676879    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676881    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676884    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676886    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676889    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676891    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676893    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676896    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676898    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676901    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676903    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676906    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676908    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676911    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676913    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676916    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:15:17.681856 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676918    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676921    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676923    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676926    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676928    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676931    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676934    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676936    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676939    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676947    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676949    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676952    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676954    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676957    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676959    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676962    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676965    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676968    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676971    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:15:17.682342 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676974    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676976    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676979    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676981    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676984    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676986    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676989    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676991    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676994    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.676996    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677000    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677003    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677006    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677008    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677011    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677014    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677016    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677020    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677023    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677026    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:15:17.682789 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677029    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677032    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677035    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677042    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677045    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677048    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677050    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:17.677053    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.677058    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.677176    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.679631    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.681325    2582 server.go:1019] "Starting client certificate rotation"
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.681452    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 11:15:17.683286 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.681502    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 11:15:17.685423 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.685408    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 11:15:17.686961 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.686937    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 11:15:17.699827 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.699805    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 07 11:15:17.702422 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.702407    2582 log.go:25] "Validated CRI v1 image API"
Jul 07 11:15:17.704002 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.703984    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 11:15:17.706551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.706533    2582 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bc341356-5b84-434b-ae08-945bed2c438e:/dev/nvme0n1p4 f765a66a-02db-4481-8991-39a9507cb4c2:/dev/nvme0n1p3]
Jul 07 11:15:17.706600 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.706551    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 11:15:17.711721 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.711612    2582 manager.go:216] Machine: {Timestamp:2026-07-07 11:15:17.710174278 +0000 UTC m=+0.373867426 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103175 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a864729984180aadf024998c4218d SystemUUID:ec2a8647-2998-4180-aadf-024998c4218d BootID:5befeb15-3769-49d1-9787-2721fc2c1cb9 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:22:2e:3a:d1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:22:2e:3a:d1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:ad:af:81:0e:2f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 11:15:17.711721 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.711720    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 11:15:17.711828 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.711797    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 11:15:17.715173 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.715156    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 11:15:17.715250 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.715164    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 11:15:17.715369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.715187    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-180.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 11:15:17.716100 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.716090    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 11:15:17.716141 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.716102    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 11:15:17.716141 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.716110    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 11:15:17.716928 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.716909    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 11:15:17.718765 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.718755    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 11:15:17.723368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.723356    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 11:15:17.723481 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.723471    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 11:15:17.723512 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.723501    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 11:15:17.723547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.723520    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 11:15:17.723547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.723531    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 11:15:17.723547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.723542    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 07 11:15:17.723650 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.723550    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 11:15:17.723948 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.723934    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 11:15:17.724012 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.723956    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 11:15:17.726400 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.726378    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 11:15:17.728841 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.728823    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 11:15:17.728895 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.728856    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 11:15:17.730313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730299    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 11:15:17.730360 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730320    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 11:15:17.730360 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730328    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 11:15:17.730360 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730333    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 11:15:17.730360 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730343    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 11:15:17.730360 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730349    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 11:15:17.730360 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730358    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 11:15:17.730524 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730363    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 11:15:17.730524 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730370    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 11:15:17.730524 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730376    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 11:15:17.730524 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730384    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 11:15:17.730524 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.730389    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 11:15:17.731247 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.731237    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 11:15:17.731247 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.731247    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 11:15:17.734546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.734521    2582 server.go:1268] "Started kubelet"
Jul 07 11:15:17.734640 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.734599    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 11:15:17.734733 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.734711    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 11:15:17.734777 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.734520    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-180.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 11:15:17.735065 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.734512    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 11:15:17.735335 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.734934    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 11:15:17.735503 ip-10-0-133-180 systemd[1]: Started Kubernetes Kubelet.
Jul 07 11:15:17.736024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.735579    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 11:15:17.736655 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.736638    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 11:15:17.737111 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.737061    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 11:15:17.742622 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.742411    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 11:15:17.745736 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.744630    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-180.ec2.internal.18bffdb1d7daf89d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-180.ec2.internal,UID:ip-10-0-133-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-180.ec2.internal,},FirstTimestamp:2026-07-07 11:15:17.734496413 +0000 UTC m=+0.398189564,LastTimestamp:2026-07-07 11:15:17.734496413 +0000 UTC m=+0.398189564,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-180.ec2.internal,}"
Jul 07 11:15:17.747593 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.747573    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 11:15:17.747593 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.747587    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 11:15:17.747735 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.747669    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-180.ec2.internal\" not found"
Jul 07 11:15:17.748352 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.748329    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 11:15:17.748437 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.748331    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 11:15:17.748437 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.748376    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 11:15:17.748536 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.748486    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 11:15:17.748536 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.748494    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 11:15:17.748622 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.748599    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 11:15:17.748622 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.748610    2582 factory.go:55] Registering systemd factory
Jul 07 11:15:17.748622 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.748616    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 07 11:15:17.749853 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.749826    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 11:15:17.749853 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.749835    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 11:15:17.750011 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.749869    2582 factory.go:153] Registering CRI-O factory
Jul 07 11:15:17.750011 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.749880    2582 factory.go:223] Registration of the crio container factory successfully
Jul 07 11:15:17.750011 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.749896    2582 factory.go:103] Registering Raw factory
Jul 07 11:15:17.750011 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.749908    2582 manager.go:1196] Started watching for new ooms in manager
Jul 07 11:15:17.751122 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.751109    2582 manager.go:349] Starting recovery of all containers
Jul 07 11:15:17.760823 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.760717    2582 manager.go:354] Recovery completed
Jul 07 11:15:17.762158 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.762130    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lcj4x"
Jul 07 11:15:17.764813 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.764802    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:15:17.769021 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.769008    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:15:17.769082 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.769032    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:15:17.769082 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.769043    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:15:17.769535 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.769514    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lcj4x"
Jul 07 11:15:17.770630 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.770614    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 11:15:17.770630 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.770629    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 11:15:17.770764 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.770640    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:15:17.770764 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.770646    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 11:15:17.771184 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.771099    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-180.ec2.internal.18bffdb1d9e9c604  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-180.ec2.internal,UID:ip-10-0-133-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-180.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-180.ec2.internal,},FirstTimestamp:2026-07-07 11:15:17.769020932 +0000 UTC m=+0.432714079,LastTimestamp:2026-07-07 11:15:17.769020932 +0000 UTC m=+0.432714079,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-180.ec2.internal,}"
Jul 07 11:15:17.771749 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.771721    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:15:17.771836 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.771782    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:15:17.771836 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.771799    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:15:17.772692 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.772679    2582 policy_none.go:49] "None policy: Start"
Jul 07 11:15:17.772766 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.772697    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 11:15:17.772766 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.772714    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 11:15:17.774825 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.774810    2582 policy_none.go:47] "Start"
Jul 07 11:15:17.809884 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.809867    2582 manager.go:337] "Starting Device Plugin manager"
Jul 07 11:15:17.830864 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.809904    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 11:15:17.830864 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.809917    2582 server.go:87] "Starting device plugin registration server"
Jul 07 11:15:17.830864 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.810220    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 11:15:17.830864 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.810232    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 11:15:17.830864 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.810353    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 11:15:17.830864 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.810423    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 11:15:17.830864 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.810432    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 11:15:17.830864 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.811205    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 11:15:17.830864 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.811263    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-180.ec2.internal\" not found"
Jul 07 11:15:17.892412 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.892381    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 11:15:17.893650 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.893626    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 11:15:17.893747 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.893671    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 11:15:17.893747 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.893692    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 11:15:17.893747 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.893723    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 11:15:17.896244 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.896223    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:15:17.911030 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.910990    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:15:17.912202 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.912177    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:15:17.912272 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.912217    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:15:17.912272 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.912230    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:15:17.912272 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.912252    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-180.ec2.internal"
Jul 07 11:15:17.924351 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.924330    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-180.ec2.internal"
Jul 07 11:15:17.924432 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.924353    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-180.ec2.internal\": node \"ip-10-0-133-180.ec2.internal\" not found"
Jul 07 11:15:17.945169 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:17.945147    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-180.ec2.internal\" not found"
Jul 07 11:15:17.994150 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.994129    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal"]
Jul 07 11:15:17.994250 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.994214    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:15:17.995773 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.995759    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:15:17.995840 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.995785    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:15:17.995840 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.995797    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:15:17.998181 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.998169    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:15:17.998334 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.998321    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:17.998372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:17.998349    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:15:18.002896 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.002877    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:15:18.002992 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.002904    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:15:18.002992 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.002914    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:15:18.002992 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.002930    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:15:18.002992 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.002950    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:15:18.002992 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.002961    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:15:18.005230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.005217    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.005272 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.005240    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:15:18.006126 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.006107    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:15:18.006201 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.006139    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:15:18.006201 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.006150    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:15:18.032679 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:18.032660    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-180.ec2.internal\" not found" node="ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.036754 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:18.036739    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-180.ec2.internal\" not found" node="ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.045434 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:18.045417    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-180.ec2.internal\" not found"
Jul 07 11:15:18.050744 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.050728    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f055528432bc701b0b32a34b3d1cf9f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal\" (UID: \"5f055528432bc701b0b32a34b3d1cf9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.050812 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.050752    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f055528432bc701b0b32a34b3d1cf9f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal\" (UID: \"5f055528432bc701b0b32a34b3d1cf9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.050812 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.050768    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/994634ad57956f8c96f482cc22f50bb3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-180.ec2.internal\" (UID: \"994634ad57956f8c96f482cc22f50bb3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.145900 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:18.145880    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-180.ec2.internal\" not found"
Jul 07 11:15:18.151289 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.151273    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f055528432bc701b0b32a34b3d1cf9f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal\" (UID: \"5f055528432bc701b0b32a34b3d1cf9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.151361 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.151294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f055528432bc701b0b32a34b3d1cf9f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal\" (UID: \"5f055528432bc701b0b32a34b3d1cf9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.151361 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.151310    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/994634ad57956f8c96f482cc22f50bb3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-180.ec2.internal\" (UID: \"994634ad57956f8c96f482cc22f50bb3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.151361 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.151349    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/994634ad57956f8c96f482cc22f50bb3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-180.ec2.internal\" (UID: \"994634ad57956f8c96f482cc22f50bb3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.151505 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.151375    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f055528432bc701b0b32a34b3d1cf9f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal\" (UID: \"5f055528432bc701b0b32a34b3d1cf9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.151505 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.151390    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f055528432bc701b0b32a34b3d1cf9f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal\" (UID: \"5f055528432bc701b0b32a34b3d1cf9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.246511 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:18.246439    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-180.ec2.internal\" not found"
Jul 07 11:15:18.335959 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.335924    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.339626 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.339597    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.347248 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:18.347230    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-180.ec2.internal\" not found"
Jul 07 11:15:18.447766 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:18.447741    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-180.ec2.internal\" not found"
Jul 07 11:15:18.548235 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:18.548151    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-180.ec2.internal\" not found"
Jul 07 11:15:18.585862 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.585842    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:15:18.647877 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.647859    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.661930 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.661914    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 11:15:18.663398 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.663385    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal"
Jul 07 11:15:18.674090 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.674076    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 11:15:18.680219 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.680204    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 11:15:18.680313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.680296    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 11:15:18.680361 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:18.680342    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 11:15:19.849429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.849411    2582 apiserver.go:52] "Watching apiserver"
Jul 07 11:15:19.866447 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.866427    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 11:15:19.867227 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.867208    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-sxqmr","openshift-multus/multus-additional-cni-plugins-m464b","openshift-multus/network-metrics-daemon-rfz9v","openshift-network-operator/iptables-alerter-m5jjb","openshift-ovn-kubernetes/ovnkube-node-6fb5x","kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal","openshift-dns/node-resolver-ksbdk","openshift-image-registry/node-ca-57cf9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal","openshift-multus/multus-k2fgc","openshift-network-diagnostics/network-check-target-524zs","kube-system/global-pull-secret-syncer-cb8jb","kube-system/konnectivity-agent-cq9jg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"]
Jul 07 11:15:19.869758 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.869741    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.871987 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.871972    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.872345 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.872324    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:15:19.872420 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.872401    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2nqkf\""
Jul 07 11:15:19.872716 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.872697    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 11:15:19.872716 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.872711    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 11:15:19.874133 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.874118    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:19.874209 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.874184    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-w2frx\""
Jul 07 11:15:19.874253 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:19.874225    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:19.874912 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.874794    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 11:15:19.874912 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.874891    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 11:15:19.876482 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.876464    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:19.877550 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.877534    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 11:15:19.877641 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.877538    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 11:15:19.877919 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.877901    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 11:15:19.878415 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.878395    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:15:19.878795 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.878779    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 11:15:19.878872 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.878816    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 11:15:19.878915 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.878886    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dh2cr\""
Jul 07 11:15:19.879050 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879037    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.879103 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879076    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-sysconfig\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879153 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879105    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bwf2\" (UniqueName: \"kubernetes.io/projected/f58a4b96-7584-4de7-b827-10efe936fecd-kube-api-access-9bwf2\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879153 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879128    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-sys\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879153 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879149    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f58a4b96-7584-4de7-b827-10efe936fecd-etc-tuned\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879315 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879174    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0266f5e8-632f-4c24-a508-abaa012b9483-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.879315 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879215    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hpgr\" (UniqueName: \"kubernetes.io/projected/0266f5e8-632f-4c24-a508-abaa012b9483-kube-api-access-6hpgr\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.879315 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879239    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkd24\" (UniqueName: \"kubernetes.io/projected/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-kube-api-access-wkd24\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:19.879315 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879262    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f58a4b96-7584-4de7-b827-10efe936fecd-tmp\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879315 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-kubernetes\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879534 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879323    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-sysctl-d\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879534 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879357    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-sysctl-conf\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879534 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879391    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-os-release\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.879534 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879413    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0266f5e8-632f-4c24-a508-abaa012b9483-cni-binary-copy\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.879534 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879439    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.879534 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879453    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-modprobe-d\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879534 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879530    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-run\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-system-cni-dir\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.879797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879583    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-cnibin\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.879797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879621    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0266f5e8-632f-4c24-a508-abaa012b9483-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.879797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879646    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-systemd\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879668    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:19.879797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879688    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a451afbe-a804-458f-aee1-7340386ca641-host-slash\") pod \"iptables-alerter-m5jjb\" (UID: \"a451afbe-a804-458f-aee1-7340386ca641\") " pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:19.879797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879706    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-var-lib-kubelet\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879722    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-host\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.879797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879751    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a451afbe-a804-458f-aee1-7340386ca641-iptables-alerter-script\") pod \"iptables-alerter-m5jjb\" (UID: \"a451afbe-a804-458f-aee1-7340386ca641\") " pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:19.879797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879781    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crtk5\" (UniqueName: \"kubernetes.io/projected/a451afbe-a804-458f-aee1-7340386ca641-kube-api-access-crtk5\") pod \"iptables-alerter-m5jjb\" (UID: \"a451afbe-a804-458f-aee1-7340386ca641\") " pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:19.880153 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.879802    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-lib-modules\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.881435 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.881420    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:19.881522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.881466    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 11:15:19.881867 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.881821    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hj7dq\""
Jul 07 11:15:19.881867 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.881861    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 11:15:19.881975 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.881941    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 11:15:19.883083 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.883065    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 11:15:19.883149 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.883118    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 11:15:19.883246 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.883065    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 11:15:19.883726 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.883700    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:19.883997 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.883856    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 11:15:19.884116 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.884101    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xw8q6\""
Jul 07 11:15:19.884159 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.884119    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 11:15:19.886071 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.886056    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.886167 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.886154    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 11:15:19.886356 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.886341    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 11:15:19.886490 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.886479    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pbvtp\""
Jul 07 11:15:19.887112 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.887099    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 11:15:19.887168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.887106    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 11:15:19.888262 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.888247    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:19.888337 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:19.888318    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:19.888672 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.888659    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tcpsg\""
Jul 07 11:15:19.888812 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.888800    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 11:15:19.890503 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.890491    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:19.890558 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:19.890543    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:19.892733 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.892718    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:19.895006 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.894991    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:19.895351 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.895309    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 11:15:19.895471 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.895457    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 11:15:19.895540 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.895511    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fpz7v\""
Jul 07 11:15:19.897278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.897245    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 11:10:18 +0000 UTC" deadline="2027-12-28 12:59:29.446150647 +0000 UTC"
Jul 07 11:15:19.897278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.897275    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12937h44m9.548877339s"
Jul 07 11:15:19.897427 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.897415    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ljtwf\""
Jul 07 11:15:19.897522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.897509    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 11:15:19.897668 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.897652    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 11:15:19.897866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.897852    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 11:15:19.939733 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.939716    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:15:19.949293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.949277    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bbmw8"
Jul 07 11:15:19.960933 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.960916    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bbmw8"
Jul 07 11:15:19.975148 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.975129    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 11:15:19.980225 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980186    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.980317 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980235    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-device-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:19.980317 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980254    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmlf4\" (UniqueName: \"kubernetes.io/projected/1113c816-17c6-479c-bb9e-36ea9b537bdb-kube-api-access-kmlf4\") pod \"node-ca-57cf9\" (UID: \"1113c816-17c6-479c-bb9e-36ea9b537bdb\") " pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:19.980317 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980270    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ba044ce-9149-4c30-b2ac-2317d0bf953b-cni-binary-copy\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.980317 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980289    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-slash\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.980317 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980308    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-var-lib-openvswitch\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980347    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980354    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-log-socket\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980399    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-modprobe-d\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980421    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-system-cni-dir\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0266f5e8-632f-4c24-a508-abaa012b9483-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980453    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-var-lib-cni-bin\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980474    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-var-lib-cni-multus\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980497    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-var-lib-kubelet\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980521    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/360ce88e-8a7a-4e3a-bd30-1d2854664c35-tmp-dir\") pod \"node-resolver-ksbdk\" (UID: \"360ce88e-8a7a-4e3a-bd30-1d2854664c35\") " pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980502    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-system-cni-dir\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.980551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980545    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980572    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-systemd\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980581    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-modprobe-d\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980596    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hd6k\" (UniqueName: \"kubernetes.io/projected/d5a8106f-d2a4-4b29-bc33-dea166f5f249-kube-api-access-6hd6k\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980616    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-systemd\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980619    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-daemon-config\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980656    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-kubelet\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980679    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-var-lib-kubelet\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-host\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980721    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a451afbe-a804-458f-aee1-7340386ca641-iptables-alerter-script\") pod \"iptables-alerter-m5jjb\" (UID: \"a451afbe-a804-458f-aee1-7340386ca641\") " pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980723    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-var-lib-kubelet\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980755    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-cnibin\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980781    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-etc-openvswitch\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980779    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-host\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980801    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-run-openvswitch\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980844    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980871    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1113c816-17c6-479c-bb9e-36ea9b537bdb-host\") pod \"node-ca-57cf9\" (UID: \"1113c816-17c6-479c-bb9e-36ea9b537bdb\") " pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:19.981155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980895    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-conf-dir\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980921    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f58a4b96-7584-4de7-b827-10efe936fecd-etc-tuned\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0266f5e8-632f-4c24-a508-abaa012b9483-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.980973    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hpgr\" (UniqueName: \"kubernetes.io/projected/0266f5e8-632f-4c24-a508-abaa012b9483-kube-api-access-6hpgr\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981000    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkd24\" (UniqueName: \"kubernetes.io/projected/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-kube-api-access-wkd24\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-registration-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981049    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-etc-selinux\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981072    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a103de2-c03c-4ba3-9c40-184af233d72a-agent-certs\") pod \"konnectivity-agent-cq9jg\" (UID: \"9a103de2-c03c-4ba3-9c40-184af233d72a\") " pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981090    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0266f5e8-632f-4c24-a508-abaa012b9483-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981096    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f58a4b96-7584-4de7-b827-10efe936fecd-tmp\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981134    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-os-release\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981162    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c651db08-9dd6-4407-957c-088300bb56a6-env-overrides\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981190    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-kubernetes\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-sysctl-conf\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981260    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-os-release\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981252    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981293    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a451afbe-a804-458f-aee1-7340386ca641-iptables-alerter-script\") pod \"iptables-alerter-m5jjb\" (UID: \"a451afbe-a804-458f-aee1-7340386ca641\") " pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:19.982099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981312    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0266f5e8-632f-4c24-a508-abaa012b9483-cni-binary-copy\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981329    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-os-release\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981343    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-socket-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981355    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-kubernetes\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981368    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-run-k8s-cni-cncf-io\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrtn7\" (UniqueName: \"kubernetes.io/projected/8ba044ce-9149-4c30-b2ac-2317d0bf953b-kube-api-access-hrtn7\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981409    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0266f5e8-632f-4c24-a508-abaa012b9483-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-run\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-cnibin\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981458    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-run\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981417    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-sysctl-conf\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981489    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-sys-fs\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981517    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0266f5e8-632f-4c24-a508-abaa012b9483-cnibin\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981562    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-hostroot\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981583    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981616    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-cni-bin\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981639    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-cni-netd\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.983085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c651db08-9dd6-4407-957c-088300bb56a6-ovnkube-config\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a451afbe-a804-458f-aee1-7340386ca641-host-slash\") pod \"iptables-alerter-m5jjb\" (UID: \"a451afbe-a804-458f-aee1-7340386ca641\") " pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981753    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-socket-dir-parent\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-run-multus-certs\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:19.981785    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981796    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a451afbe-a804-458f-aee1-7340386ca641-host-slash\") pod \"iptables-alerter-m5jjb\" (UID: \"a451afbe-a804-458f-aee1-7340386ca641\") " pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/360ce88e-8a7a-4e3a-bd30-1d2854664c35-hosts-file\") pod \"node-resolver-ksbdk\" (UID: \"360ce88e-8a7a-4e3a-bd30-1d2854664c35\") " pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981852    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0266f5e8-632f-4c24-a508-abaa012b9483-cni-binary-copy\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:19.981864    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs podName:0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca nodeName:}" failed. No retries permitted until 2026-07-07 11:15:20.481830362 +0000 UTC m=+2.020521754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs") pod "network-metrics-daemon-rfz9v" (UID: "0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981884    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6fjv\" (UniqueName: \"kubernetes.io/projected/360ce88e-8a7a-4e3a-bd30-1d2854664c35-kube-api-access-r6fjv\") pod \"node-resolver-ksbdk\" (UID: \"360ce88e-8a7a-4e3a-bd30-1d2854664c35\") " pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crtk5\" (UniqueName: \"kubernetes.io/projected/a451afbe-a804-458f-aee1-7340386ca641-kube-api-access-crtk5\") pod \"iptables-alerter-m5jjb\" (UID: \"a451afbe-a804-458f-aee1-7340386ca641\") " pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981928    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-run-systemd\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981951    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-run-ovn\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981967    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-lib-modules\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.981987    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-run-netns\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982008    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a103de2-c03c-4ba3-9c40-184af233d72a-konnectivity-ca\") pod \"konnectivity-agent-cq9jg\" (UID: \"9a103de2-c03c-4ba3-9c40-184af233d72a\") " pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:19.983964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/96412d1b-55ef-4558-88c3-4b83911151b5-dbus\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982040    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-sysconfig\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bwf2\" (UniqueName: \"kubernetes.io/projected/f58a4b96-7584-4de7-b827-10efe936fecd-kube-api-access-9bwf2\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982071    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1113c816-17c6-479c-bb9e-36ea9b537bdb-serviceca\") pod \"node-ca-57cf9\" (UID: \"1113c816-17c6-479c-bb9e-36ea9b537bdb\") " pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982090    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-run-netns\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982106    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-sysconfig\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982110    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr2rf\" (UniqueName: \"kubernetes.io/projected/c651db08-9dd6-4407-957c-088300bb56a6-kube-api-access-hr2rf\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982068    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-lib-modules\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982149    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/96412d1b-55ef-4558-88c3-4b83911151b5-kubelet-config\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982237    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-sys\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982257    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-sys\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982289    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-systemd-units\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982313    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c651db08-9dd6-4407-957c-088300bb56a6-ovn-node-metrics-cert\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982340    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-system-cni-dir\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982363    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-cni-dir\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982385    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-etc-kubernetes\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:19.984816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982410    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ldcr\" (UniqueName: \"kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr\") pod \"network-check-target-524zs\" (UID: \"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2\") " pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:19.985746 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982434    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-node-log\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.985746 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982459    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c651db08-9dd6-4407-957c-088300bb56a6-ovnkube-script-lib\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:19.985746 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982486    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-sysctl-d\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.985746 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.982601    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f58a4b96-7584-4de7-b827-10efe936fecd-etc-sysctl-d\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.985746 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.983985    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f58a4b96-7584-4de7-b827-10efe936fecd-etc-tuned\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.985746 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.984029    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f58a4b96-7584-4de7-b827-10efe936fecd-tmp\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.994303 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.994285    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkd24\" (UniqueName: \"kubernetes.io/projected/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-kube-api-access-wkd24\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:19.994372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.994303    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bwf2\" (UniqueName: \"kubernetes.io/projected/f58a4b96-7584-4de7-b827-10efe936fecd-kube-api-access-9bwf2\") pod \"tuned-sxqmr\" (UID: \"f58a4b96-7584-4de7-b827-10efe936fecd\") " pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:19.994372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.994308    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crtk5\" (UniqueName: \"kubernetes.io/projected/a451afbe-a804-458f-aee1-7340386ca641-kube-api-access-crtk5\") pod \"iptables-alerter-m5jjb\" (UID: \"a451afbe-a804-458f-aee1-7340386ca641\") " pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:19.994372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:19.994320    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hpgr\" (UniqueName: \"kubernetes.io/projected/0266f5e8-632f-4c24-a508-abaa012b9483-kube-api-access-6hpgr\") pod \"multus-additional-cni-plugins-m464b\" (UID: \"0266f5e8-632f-4c24-a508-abaa012b9483\") " pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:20.083486 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083459    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-run-netns\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.083658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a103de2-c03c-4ba3-9c40-184af233d72a-konnectivity-ca\") pod \"konnectivity-agent-cq9jg\" (UID: \"9a103de2-c03c-4ba3-9c40-184af233d72a\") " pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:20.083658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083517    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-run-netns\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.083658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083529    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/96412d1b-55ef-4558-88c3-4b83911151b5-dbus\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:20.083658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083578    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1113c816-17c6-479c-bb9e-36ea9b537bdb-serviceca\") pod \"node-ca-57cf9\" (UID: \"1113c816-17c6-479c-bb9e-36ea9b537bdb\") " pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:20.083658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-run-netns\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.083658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr2rf\" (UniqueName: \"kubernetes.io/projected/c651db08-9dd6-4407-957c-088300bb56a6-kube-api-access-hr2rf\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.083658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083652    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/96412d1b-55ef-4558-88c3-4b83911151b5-kubelet-config\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083627    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/96412d1b-55ef-4558-88c3-4b83911151b5-dbus\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083673    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-run-netns\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083678    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083730    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/96412d1b-55ef-4558-88c3-4b83911151b5-kubelet-config\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083755    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083798    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-systemd-units\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c651db08-9dd6-4407-957c-088300bb56a6-ovn-node-metrics-cert\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083860    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-system-cni-dir\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083878    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-systemd-units\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083883    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-cni-dir\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083932    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-system-cni-dir\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083943    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-etc-kubernetes\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.083983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083960    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-cni-dir\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.083972    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldcr\" (UniqueName: \"kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr\") pod \"network-check-target-524zs\" (UID: \"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2\") " pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084012    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-etc-kubernetes\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084014    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a103de2-c03c-4ba3-9c40-184af233d72a-konnectivity-ca\") pod \"konnectivity-agent-cq9jg\" (UID: \"9a103de2-c03c-4ba3-9c40-184af233d72a\") " pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084022    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-node-log\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084006    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1113c816-17c6-479c-bb9e-36ea9b537bdb-serviceca\") pod \"node-ca-57cf9\" (UID: \"1113c816-17c6-479c-bb9e-36ea9b537bdb\") " pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c651db08-9dd6-4407-957c-088300bb56a6-ovnkube-script-lib\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084073    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-device-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084085    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-node-log\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084099    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmlf4\" (UniqueName: \"kubernetes.io/projected/1113c816-17c6-479c-bb9e-36ea9b537bdb-kube-api-access-kmlf4\") pod \"node-ca-57cf9\" (UID: \"1113c816-17c6-479c-bb9e-36ea9b537bdb\") " pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084122    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ba044ce-9149-4c30-b2ac-2317d0bf953b-cni-binary-copy\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084133    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-device-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-slash\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-var-lib-openvswitch\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084235    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-log-socket\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084244    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-slash\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084266    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-var-lib-cni-bin\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084268    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-var-lib-openvswitch\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.084565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-var-lib-cni-multus\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084298    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-log-socket\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-var-lib-kubelet\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084320    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-var-lib-cni-bin\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084336    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-var-lib-cni-multus\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084350    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/360ce88e-8a7a-4e3a-bd30-1d2854664c35-tmp-dir\") pod \"node-resolver-ksbdk\" (UID: \"360ce88e-8a7a-4e3a-bd30-1d2854664c35\") " pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084366    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-var-lib-kubelet\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084375    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084402    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hd6k\" (UniqueName: \"kubernetes.io/projected/d5a8106f-d2a4-4b29-bc33-dea166f5f249-kube-api-access-6hd6k\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-daemon-config\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084448    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-kubelet\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084475    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-cnibin\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.084496    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084535    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-etc-openvswitch\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.084550    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret podName:96412d1b-55ef-4558-88c3-4b83911151b5 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:20.584532236 +0000 UTC m=+2.123223626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret") pod "global-pull-secret-syncer-cb8jb" (UID: "96412d1b-55ef-4558-88c3-4b83911151b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-etc-openvswitch\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-run-openvswitch\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.085374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084632    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084645    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/360ce88e-8a7a-4e3a-bd30-1d2854664c35-tmp-dir\") pod \"node-resolver-ksbdk\" (UID: \"360ce88e-8a7a-4e3a-bd30-1d2854664c35\") " pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084658    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1113c816-17c6-479c-bb9e-36ea9b537bdb-host\") pod \"node-ca-57cf9\" (UID: \"1113c816-17c6-479c-bb9e-36ea9b537bdb\") " pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084645    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c651db08-9dd6-4407-957c-088300bb56a6-ovnkube-script-lib\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084686    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-kubelet\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084689    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-conf-dir\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084728    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-run-openvswitch\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084727    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-registration-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084747    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8ba044ce-9149-4c30-b2ac-2317d0bf953b-cni-binary-copy\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084767    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1113c816-17c6-479c-bb9e-36ea9b537bdb-host\") pod \"node-ca-57cf9\" (UID: \"1113c816-17c6-479c-bb9e-36ea9b537bdb\") " pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084773    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-conf-dir\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084773    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084732    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-cnibin\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-etc-selinux\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084818    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a103de2-c03c-4ba3-9c40-184af233d72a-agent-certs\") pod \"konnectivity-agent-cq9jg\" (UID: \"9a103de2-c03c-4ba3-9c40-184af233d72a\") " pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084822    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-registration-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-os-release\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084871    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c651db08-9dd6-4407-957c-088300bb56a6-env-overrides\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-socket-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084915    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-etc-selinux\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084928    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-run-k8s-cni-cncf-io\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084913    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-os-release\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084929    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-daemon-config\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084964    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-run-k8s-cni-cncf-io\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.084988    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrtn7\" (UniqueName: \"kubernetes.io/projected/8ba044ce-9149-4c30-b2ac-2317d0bf953b-kube-api-access-hrtn7\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085031    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-sys-fs\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085044    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-socket-dir\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085055    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-hostroot\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085087    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085091    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-hostroot\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085134    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-cni-bin\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085135    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5a8106f-d2a4-4b29-bc33-dea166f5f249-sys-fs\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-cni-netd\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085171    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085182    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c651db08-9dd6-4407-957c-088300bb56a6-ovnkube-config\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085220    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-cni-bin\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.086908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085222    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-host-cni-netd\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085266    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-socket-dir-parent\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-run-multus-certs\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085319    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/360ce88e-8a7a-4e3a-bd30-1d2854664c35-hosts-file\") pod \"node-resolver-ksbdk\" (UID: \"360ce88e-8a7a-4e3a-bd30-1d2854664c35\") " pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085322    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c651db08-9dd6-4407-957c-088300bb56a6-env-overrides\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085335    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-multus-socket-dir-parent\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085344    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6fjv\" (UniqueName: \"kubernetes.io/projected/360ce88e-8a7a-4e3a-bd30-1d2854664c35-kube-api-access-r6fjv\") pod \"node-resolver-ksbdk\" (UID: \"360ce88e-8a7a-4e3a-bd30-1d2854664c35\") " pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085372    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-run-systemd\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085379    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8ba044ce-9149-4c30-b2ac-2317d0bf953b-host-run-multus-certs\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085394    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-run-ovn\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085425    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/360ce88e-8a7a-4e3a-bd30-1d2854664c35-hosts-file\") pod \"node-resolver-ksbdk\" (UID: \"360ce88e-8a7a-4e3a-bd30-1d2854664c35\") " pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085511    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-run-ovn\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.085851    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c651db08-9dd6-4407-957c-088300bb56a6-run-systemd\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.087863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.086295    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c651db08-9dd6-4407-957c-088300bb56a6-ovnkube-config\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.088827 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.088799    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c651db08-9dd6-4407-957c-088300bb56a6-ovn-node-metrics-cert\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.089711 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.089689    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a103de2-c03c-4ba3-9c40-184af233d72a-agent-certs\") pod \"konnectivity-agent-cq9jg\" (UID: \"9a103de2-c03c-4ba3-9c40-184af233d72a\") " pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:20.097499 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.097479    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:15:20.097499 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.097501    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:15:20.097684 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.097511    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4ldcr for pod openshift-network-diagnostics/network-check-target-524zs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:20.097684 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.097574    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr podName:4c33a0a8-99d2-4472-8aac-3b9cd92eaee2 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:20.597558062 +0000 UTC m=+2.136249460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4ldcr" (UniqueName: "kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr") pod "network-check-target-524zs" (UID: "4c33a0a8-99d2-4472-8aac-3b9cd92eaee2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:20.099610 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.099580    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrtn7\" (UniqueName: \"kubernetes.io/projected/8ba044ce-9149-4c30-b2ac-2317d0bf953b-kube-api-access-hrtn7\") pod \"multus-k2fgc\" (UID: \"8ba044ce-9149-4c30-b2ac-2317d0bf953b\") " pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.099853 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.099834    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hd6k\" (UniqueName: \"kubernetes.io/projected/d5a8106f-d2a4-4b29-bc33-dea166f5f249-kube-api-access-6hd6k\") pod \"aws-ebs-csi-driver-node-mdq6v\" (UID: \"d5a8106f-d2a4-4b29-bc33-dea166f5f249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.099853 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.099848    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6fjv\" (UniqueName: \"kubernetes.io/projected/360ce88e-8a7a-4e3a-bd30-1d2854664c35-kube-api-access-r6fjv\") pod \"node-resolver-ksbdk\" (UID: \"360ce88e-8a7a-4e3a-bd30-1d2854664c35\") " pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:20.099944 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.099921    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmlf4\" (UniqueName: \"kubernetes.io/projected/1113c816-17c6-479c-bb9e-36ea9b537bdb-kube-api-access-kmlf4\") pod \"node-ca-57cf9\" (UID: \"1113c816-17c6-479c-bb9e-36ea9b537bdb\") " pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:20.100348 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.100325    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr2rf\" (UniqueName: \"kubernetes.io/projected/c651db08-9dd6-4407-957c-088300bb56a6-kube-api-access-hr2rf\") pod \"ovnkube-node-6fb5x\" (UID: \"c651db08-9dd6-4407-957c-088300bb56a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.133933 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.133903    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f055528432bc701b0b32a34b3d1cf9f.slice/crio-974ad4aac60e12c151ff4ef0678b181fea0f3354402bfaf474eb89bf521f7442 WatchSource:0}: Error finding container 974ad4aac60e12c151ff4ef0678b181fea0f3354402bfaf474eb89bf521f7442: Status 404 returned error can't find the container with id 974ad4aac60e12c151ff4ef0678b181fea0f3354402bfaf474eb89bf521f7442
Jul 07 11:15:20.139063 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.139044    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod994634ad57956f8c96f482cc22f50bb3.slice/crio-96fdd6d49ae6a41707569ac8150dd177e4afa99cd7a065ba9af324ec1297cf0d WatchSource:0}: Error finding container 96fdd6d49ae6a41707569ac8150dd177e4afa99cd7a065ba9af324ec1297cf0d: Status 404 returned error can't find the container with id 96fdd6d49ae6a41707569ac8150dd177e4afa99cd7a065ba9af324ec1297cf0d
Jul 07 11:15:20.139995 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.139978    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:15:20.199188 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.199127    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sxqmr"
Jul 07 11:15:20.208422 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.208399    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58a4b96_7584_4de7_b827_10efe936fecd.slice/crio-d368cbe2228877f6ea271675c3027e58c92b234c5b2a4834a9e4f210c1af41b6 WatchSource:0}: Error finding container d368cbe2228877f6ea271675c3027e58c92b234c5b2a4834a9e4f210c1af41b6: Status 404 returned error can't find the container with id d368cbe2228877f6ea271675c3027e58c92b234c5b2a4834a9e4f210c1af41b6
Jul 07 11:15:20.211751 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.211727    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:15:20.227990 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.227972    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m464b"
Jul 07 11:15:20.235739 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.235717    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0266f5e8_632f_4c24_a508_abaa012b9483.slice/crio-05fa80ca9a1cafccd5aa2136376c22a9abfaa4766632574b985129eb5044d3f6 WatchSource:0}: Error finding container 05fa80ca9a1cafccd5aa2136376c22a9abfaa4766632574b985129eb5044d3f6: Status 404 returned error can't find the container with id 05fa80ca9a1cafccd5aa2136376c22a9abfaa4766632574b985129eb5044d3f6
Jul 07 11:15:20.241545 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.241529    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m5jjb"
Jul 07 11:15:20.246372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.246349    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:20.250645 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.250623    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda451afbe_a804_458f_aee1_7340386ca641.slice/crio-c74b59a130e11285f9cc933542a4b64230e9cdb9b14220d2af471a3d04c1ba41 WatchSource:0}: Error finding container c74b59a130e11285f9cc933542a4b64230e9cdb9b14220d2af471a3d04c1ba41: Status 404 returned error can't find the container with id c74b59a130e11285f9cc933542a4b64230e9cdb9b14220d2af471a3d04c1ba41
Jul 07 11:15:20.254772 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.254752    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc651db08_9dd6_4407_957c_088300bb56a6.slice/crio-f56f212a1532067611fa2ee7b8317db608b680dd141aa6aec76f5e620e8be55f WatchSource:0}: Error finding container f56f212a1532067611fa2ee7b8317db608b680dd141aa6aec76f5e620e8be55f: Status 404 returned error can't find the container with id f56f212a1532067611fa2ee7b8317db608b680dd141aa6aec76f5e620e8be55f
Jul 07 11:15:20.278387 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.278371    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ksbdk"
Jul 07 11:15:20.286574 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.286546    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod360ce88e_8a7a_4e3a_bd30_1d2854664c35.slice/crio-d306447a9ab932d85af322abc6c1e91d8b837e724de4a02e5cb81b44305fd19b WatchSource:0}: Error finding container d306447a9ab932d85af322abc6c1e91d8b837e724de4a02e5cb81b44305fd19b: Status 404 returned error can't find the container with id d306447a9ab932d85af322abc6c1e91d8b837e724de4a02e5cb81b44305fd19b
Jul 07 11:15:20.289962 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.289946    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-57cf9"
Jul 07 11:15:20.297534 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.297513    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1113c816_17c6_479c_bb9e_36ea9b537bdb.slice/crio-622ce7d8ab0802e1c5a31cfc2526db940ac1c6913557c8142e367a7632a99a43 WatchSource:0}: Error finding container 622ce7d8ab0802e1c5a31cfc2526db940ac1c6913557c8142e367a7632a99a43: Status 404 returned error can't find the container with id 622ce7d8ab0802e1c5a31cfc2526db940ac1c6913557c8142e367a7632a99a43
Jul 07 11:15:20.310826 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.310810    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k2fgc"
Jul 07 11:15:20.317990 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.317971    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba044ce_9149_4c30_b2ac_2317d0bf953b.slice/crio-141e4348fed635c87923290b2cc9bed8f31091f76981d4f393f9b035a602741b WatchSource:0}: Error finding container 141e4348fed635c87923290b2cc9bed8f31091f76981d4f393f9b035a602741b: Status 404 returned error can't find the container with id 141e4348fed635c87923290b2cc9bed8f31091f76981d4f393f9b035a602741b
Jul 07 11:15:20.321883 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.321868    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:20.325549 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.325535    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v"
Jul 07 11:15:20.329588 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.329569    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a103de2_c03c_4ba3_9c40_184af233d72a.slice/crio-629f770ea1681bf749d95e82591bcdf72c468aea9eaac1db55c183cdc1678417 WatchSource:0}: Error finding container 629f770ea1681bf749d95e82591bcdf72c468aea9eaac1db55c183cdc1678417: Status 404 returned error can't find the container with id 629f770ea1681bf749d95e82591bcdf72c468aea9eaac1db55c183cdc1678417
Jul 07 11:15:20.333593 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:20.333573    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a8106f_d2a4_4b29_bc33_dea166f5f249.slice/crio-5a3a8cff4c89219892f14b90270c4163cb3febcb3edba03a1f2010261499e0f2 WatchSource:0}: Error finding container 5a3a8cff4c89219892f14b90270c4163cb3febcb3edba03a1f2010261499e0f2: Status 404 returned error can't find the container with id 5a3a8cff4c89219892f14b90270c4163cb3febcb3edba03a1f2010261499e0f2
Jul 07 11:15:20.335705 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.335685    2582 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-driver start failed in pod aws-ebs-csi-driver-node-mdq6v_openshift-cluster-csi-drivers(d5a8106f-d2a4-4b29-bc33-dea166f5f249): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 07 11:15:20.336549 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.336531    2582 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-node-driver-registrar start failed in pod aws-ebs-csi-driver-node-mdq6v_openshift-cluster-csi-drivers(d5a8106f-d2a4-4b29-bc33-dea166f5f249): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 07 11:15:20.338330 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.338314    2582 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-liveness-probe start failed in pod aws-ebs-csi-driver-node-mdq6v_openshift-cluster-csi-drivers(d5a8106f-d2a4-4b29-bc33-dea166f5f249): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 07 11:15:20.338371 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.338347    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v" podUID="d5a8106f-d2a4-4b29-bc33-dea166f5f249"
Jul 07 11:15:20.487923 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.487892    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:20.488061 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.488046    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:20.488107 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.488098    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs podName:0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca nodeName:}" failed. No retries permitted until 2026-07-07 11:15:21.488084038 +0000 UTC m=+3.026775431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs") pod "network-metrics-daemon-rfz9v" (UID: "0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:20.588424 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.588400    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:20.588579 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.588553    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:20.588645 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.588618    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret podName:96412d1b-55ef-4558-88c3-4b83911151b5 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:21.58859869 +0000 UTC m=+3.127290097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret") pod "global-pull-secret-syncer-cb8jb" (UID: "96412d1b-55ef-4558-88c3-4b83911151b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:20.689575 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.689249    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldcr\" (UniqueName: \"kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr\") pod \"network-check-target-524zs\" (UID: \"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2\") " pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:20.689575 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.689427    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:15:20.689575 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.689448    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:15:20.689575 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.689461    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4ldcr for pod openshift-network-diagnostics/network-check-target-524zs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:20.689575 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:20.689521    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr podName:4c33a0a8-99d2-4472-8aac-3b9cd92eaee2 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:21.689503692 +0000 UTC m=+3.228195101 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4ldcr" (UniqueName: "kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr") pod "network-check-target-524zs" (UID: "4c33a0a8-99d2-4472-8aac-3b9cd92eaee2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:20.919206 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.918936    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:15:20.964489 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.964168    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:10:19 +0000 UTC" deadline="2028-01-12 20:47:59.774933241 +0000 UTC"
Jul 07 11:15:20.964489 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:20.964433    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13305h32m38.810507698s"
Jul 07 11:15:21.020554 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.020526    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:21.020708 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:21.020668    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:21.057959 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.057900    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal" event={"ID":"994634ad57956f8c96f482cc22f50bb3","Type":"ContainerStarted","Data":"96fdd6d49ae6a41707569ac8150dd177e4afa99cd7a065ba9af324ec1297cf0d"}
Jul 07 11:15:21.078936 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.078900    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cq9jg" event={"ID":"9a103de2-c03c-4ba3-9c40-184af233d72a","Type":"ContainerStarted","Data":"629f770ea1681bf749d95e82591bcdf72c468aea9eaac1db55c183cdc1678417"}
Jul 07 11:15:21.084955 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.084926    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ksbdk" event={"ID":"360ce88e-8a7a-4e3a-bd30-1d2854664c35","Type":"ContainerStarted","Data":"d306447a9ab932d85af322abc6c1e91d8b837e724de4a02e5cb81b44305fd19b"}
Jul 07 11:15:21.088777 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.088748    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x" event={"ID":"c651db08-9dd6-4407-957c-088300bb56a6","Type":"ContainerStarted","Data":"f56f212a1532067611fa2ee7b8317db608b680dd141aa6aec76f5e620e8be55f"}
Jul 07 11:15:21.102679 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.102651    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m5jjb" event={"ID":"a451afbe-a804-458f-aee1-7340386ca641","Type":"ContainerStarted","Data":"c74b59a130e11285f9cc933542a4b64230e9cdb9b14220d2af471a3d04c1ba41"}
Jul 07 11:15:21.113576 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.113481    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sxqmr" event={"ID":"f58a4b96-7584-4de7-b827-10efe936fecd","Type":"ContainerStarted","Data":"d368cbe2228877f6ea271675c3027e58c92b234c5b2a4834a9e4f210c1af41b6"}
Jul 07 11:15:21.119721 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.119693    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal" event={"ID":"5f055528432bc701b0b32a34b3d1cf9f","Type":"ContainerStarted","Data":"974ad4aac60e12c151ff4ef0678b181fea0f3354402bfaf474eb89bf521f7442"}
Jul 07 11:15:21.129109 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.129083    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v" event={"ID":"d5a8106f-d2a4-4b29-bc33-dea166f5f249","Type":"ContainerStarted","Data":"5a3a8cff4c89219892f14b90270c4163cb3febcb3edba03a1f2010261499e0f2"}
Jul 07 11:15:21.136426 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:21.136371    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8cde2dd29b991fab261b34481407faace228f79f1e92bdeef6c97eec1af7538e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aa590e7fc5391b1007aa63e4e4488de68ce3695cc319d50a3f9b438deda90b29\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:57b61ec1af2b592c8217f770864d5d3ffb8f77f17e1922158c3fab15728d6aa3\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v" podUID="d5a8106f-d2a4-4b29-bc33-dea166f5f249"
Jul 07 11:15:21.138147 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.138121    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k2fgc" event={"ID":"8ba044ce-9149-4c30-b2ac-2317d0bf953b","Type":"ContainerStarted","Data":"141e4348fed635c87923290b2cc9bed8f31091f76981d4f393f9b035a602741b"}
Jul 07 11:15:21.155787 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.155760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-57cf9" event={"ID":"1113c816-17c6-479c-bb9e-36ea9b537bdb","Type":"ContainerStarted","Data":"622ce7d8ab0802e1c5a31cfc2526db940ac1c6913557c8142e367a7632a99a43"}
Jul 07 11:15:21.160978 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.160953    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m464b" event={"ID":"0266f5e8-632f-4c24-a508-abaa012b9483","Type":"ContainerStarted","Data":"05fa80ca9a1cafccd5aa2136376c22a9abfaa4766632574b985129eb5044d3f6"}
Jul 07 11:15:21.494641 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.494604    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:21.494842 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:21.494825    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:21.494903 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:21.494889    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs podName:0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca nodeName:}" failed. No retries permitted until 2026-07-07 11:15:23.494869374 +0000 UTC m=+5.033560768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs") pod "network-metrics-daemon-rfz9v" (UID: "0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:21.595491 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.595456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:21.595670 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:21.595622    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:21.595735 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:21.595686    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret podName:96412d1b-55ef-4558-88c3-4b83911151b5 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:23.59566089 +0000 UTC m=+5.134352292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret") pod "global-pull-secret-syncer-cb8jb" (UID: "96412d1b-55ef-4558-88c3-4b83911151b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:21.696830 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.696796    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldcr\" (UniqueName: \"kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr\") pod \"network-check-target-524zs\" (UID: \"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2\") " pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:21.697006 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:21.696975    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:15:21.697006 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:21.696996    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:15:21.697006 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:21.697008    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4ldcr for pod openshift-network-diagnostics/network-check-target-524zs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:21.697168 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:21.697066    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr podName:4c33a0a8-99d2-4472-8aac-3b9cd92eaee2 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:23.697048704 +0000 UTC m=+5.235740096 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4ldcr" (UniqueName: "kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr") pod "network-check-target-524zs" (UID: "4c33a0a8-99d2-4472-8aac-3b9cd92eaee2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:21.966105 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.965955    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:10:19 +0000 UTC" deadline="2028-02-14 07:43:26.246345287 +0000 UTC"
Jul 07 11:15:21.966105 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:21.966014    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14084h28m4.280335824s"
Jul 07 11:15:22.019895 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:22.019866    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:22.020063 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:22.019993    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:22.020429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:22.020408    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:22.020527 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:22.020513    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:22.165620 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:22.165403    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:15:23.021839 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:23.021801    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:23.022331 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:23.021936    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:23.513824 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:23.513793    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:23.514001 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:23.513949    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:23.514066 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:23.514021    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs podName:0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca nodeName:}" failed. No retries permitted until 2026-07-07 11:15:27.514001172 +0000 UTC m=+9.052692580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs") pod "network-metrics-daemon-rfz9v" (UID: "0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:23.614568 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:23.614534    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:23.614727 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:23.614703    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:23.614784 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:23.614771    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret podName:96412d1b-55ef-4558-88c3-4b83911151b5 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:27.614752344 +0000 UTC m=+9.153443737 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret") pod "global-pull-secret-syncer-cb8jb" (UID: "96412d1b-55ef-4558-88c3-4b83911151b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:23.715744 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:23.715710    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldcr\" (UniqueName: \"kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr\") pod \"network-check-target-524zs\" (UID: \"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2\") " pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:23.715916 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:23.715876    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:15:23.715916 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:23.715903    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:15:23.715916 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:23.715917    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4ldcr for pod openshift-network-diagnostics/network-check-target-524zs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:23.716083 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:23.715987    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr podName:4c33a0a8-99d2-4472-8aac-3b9cd92eaee2 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:27.715967158 +0000 UTC m=+9.254658551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4ldcr" (UniqueName: "kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr") pod "network-check-target-524zs" (UID: "4c33a0a8-99d2-4472-8aac-3b9cd92eaee2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:24.019238 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:24.019207    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:24.019413 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:24.019334    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:24.019709 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:24.019688    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:24.019815 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:24.019794    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:25.019871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:25.019591    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:25.019871 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:25.019737    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:26.019683 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:26.019653    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:26.019865 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:26.019655    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:26.019865 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:26.019770    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:26.020276 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:26.019882    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:27.019108 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:27.019070    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:27.019298 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:27.019242    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:27.546931 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:27.546821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:27.547426 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:27.546948    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:27.547426 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:27.547035    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs podName:0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca nodeName:}" failed. No retries permitted until 2026-07-07 11:15:35.547011262 +0000 UTC m=+17.085702688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs") pod "network-metrics-daemon-rfz9v" (UID: "0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:27.648009 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:27.647920    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:27.648151 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:27.648057    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:27.648151 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:27.648131    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret podName:96412d1b-55ef-4558-88c3-4b83911151b5 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:35.64811109 +0000 UTC m=+17.186802486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret") pod "global-pull-secret-syncer-cb8jb" (UID: "96412d1b-55ef-4558-88c3-4b83911151b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:27.748754 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:27.748704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldcr\" (UniqueName: \"kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr\") pod \"network-check-target-524zs\" (UID: \"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2\") " pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:27.748929 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:27.748900    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:15:27.748929 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:27.748926    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:15:27.749034 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:27.748939    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4ldcr for pod openshift-network-diagnostics/network-check-target-524zs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:27.749034 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:27.749009    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr podName:4c33a0a8-99d2-4472-8aac-3b9cd92eaee2 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:35.7489879 +0000 UTC m=+17.287679292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4ldcr" (UniqueName: "kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr") pod "network-check-target-524zs" (UID: "4c33a0a8-99d2-4472-8aac-3b9cd92eaee2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:28.019165 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:28.019127    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:28.019330 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:28.019128    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:28.019330 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:28.019278    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:28.019330 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:28.019316    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:29.023131 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:29.023098    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:29.023555 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:29.023247    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:30.019295 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:30.019213    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:30.019295 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:30.019267    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:30.019566 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:30.019339    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:30.019566 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:30.019476    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:31.021965 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:31.021933    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:31.022380 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:31.022060    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:32.019210 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:32.019177    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:32.019210 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:32.019177    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:32.019452 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:32.019317    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:32.019452 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:32.019436    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:33.022469 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:33.022438    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:33.022913 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:33.022565    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:34.019729 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:34.019692    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:34.019916 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:34.019775    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:34.019983 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:34.019940    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:34.020073 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:34.020039    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:35.019355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:35.019318    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:35.019778 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:35.019464    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:35.608056 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:35.608020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:35.608260 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:35.608151    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:35.608260 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:35.608237    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs podName:0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca nodeName:}" failed. No retries permitted until 2026-07-07 11:15:51.608216015 +0000 UTC m=+33.146907496 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs") pod "network-metrics-daemon-rfz9v" (UID: "0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:35.709105 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:35.709042    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:35.709282 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:35.709164    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:35.709282 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:35.709236    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret podName:96412d1b-55ef-4558-88c3-4b83911151b5 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:51.709217173 +0000 UTC m=+33.247908567 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret") pod "global-pull-secret-syncer-cb8jb" (UID: "96412d1b-55ef-4558-88c3-4b83911151b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:35.809746 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:35.809718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldcr\" (UniqueName: \"kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr\") pod \"network-check-target-524zs\" (UID: \"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2\") " pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:35.809929 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:35.809904    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:15:35.809929 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:35.809925    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:15:35.810035 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:35.809938    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4ldcr for pod openshift-network-diagnostics/network-check-target-524zs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:35.810035 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:35.810003    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr podName:4c33a0a8-99d2-4472-8aac-3b9cd92eaee2 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:51.809984879 +0000 UTC m=+33.348676287 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4ldcr" (UniqueName: "kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr") pod "network-check-target-524zs" (UID: "4c33a0a8-99d2-4472-8aac-3b9cd92eaee2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:36.018900 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:36.018871    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:36.019067 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:36.018877    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:36.019067 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:36.018973    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:36.019169 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:36.019094    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:37.019346 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:37.019310    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:37.019752 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:37.019420    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:38.019662 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:38.019624    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:38.020078 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:38.019625    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:38.020078 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:38.019753    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:38.020078 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:38.019854    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:39.019444 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:39.019416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:39.019590 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:39.019506    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:40.020136 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:40.019867    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:40.020486 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:40.020144    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:40.020486 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:40.020182    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:40.020486 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:40.020300    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:40.205445 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:40.205400    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x" event={"ID":"c651db08-9dd6-4407-957c-088300bb56a6","Type":"ContainerStarted","Data":"dfb13b0e8791c1694fca3478371a3ae76751269db5ee3b8b0a2a97330b829c67"}
Jul 07 11:15:40.205538 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:40.205497    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x" event={"ID":"c651db08-9dd6-4407-957c-088300bb56a6","Type":"ContainerStarted","Data":"1abe35905ec68f5d18aeaf5c9fecbdf1b95221c7de9e96238f4fd37ce06814eb"}
Jul 07 11:15:40.207115 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:40.207082    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sxqmr" event={"ID":"f58a4b96-7584-4de7-b827-10efe936fecd","Type":"ContainerStarted","Data":"59750af3895ed27b909f9c0f2828fb36f3c59b65c7626c83d1b941810f1a2afc"}
Jul 07 11:15:40.209540 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:40.209374    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k2fgc" event={"ID":"8ba044ce-9149-4c30-b2ac-2317d0bf953b","Type":"ContainerStarted","Data":"0e75d76036eb5b12f81a62430f86655c2bf41ffe5259c151d27236a9243fa19c"}
Jul 07 11:15:40.217982 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:40.216245    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal" event={"ID":"994634ad57956f8c96f482cc22f50bb3","Type":"ContainerStarted","Data":"741d4fa6de914300534934bac3731ae7dbce952dfe842532e9c577d2fea2f187"}
Jul 07 11:15:40.225536 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:40.225473    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sxqmr" podStartSLOduration=1.560790731 podStartE2EDuration="21.225455435s" totalImagesPullingTime="19.664664704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:40.224990305 +0000 UTC m=+21.763681733" watchObservedRunningTime="2026-07-07 11:15:40.225455435 +0000 UTC m=+21.764146849"
Jul 07 11:15:40.268233 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:40.267934    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-180.ec2.internal" podStartSLOduration=21.267912866 podStartE2EDuration="21.267912866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:40.266693331 +0000 UTC m=+21.805384740" watchObservedRunningTime="2026-07-07 11:15:40.267912866 +0000 UTC m=+21.806604279"
Jul 07 11:15:40.268672 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:40.268619    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k2fgc" podStartSLOduration=1.71150443 podStartE2EDuration="21.26860633s" totalImagesPullingTime="19.5571019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:40.245166307 +0000 UTC m=+21.783857720" watchObservedRunningTime="2026-07-07 11:15:40.26860633 +0000 UTC m=+21.807297743"
Jul 07 11:15:41.019934 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.019906    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:41.020069 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:41.020010    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:41.219641 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.219603    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cq9jg" event={"ID":"9a103de2-c03c-4ba3-9c40-184af233d72a","Type":"ContainerStarted","Data":"391ff54062b883de87b1e8dbb08269971bc5a846e783b8c7b074077246bd1409"}
Jul 07 11:15:41.221321 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.221255    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ksbdk" event={"ID":"360ce88e-8a7a-4e3a-bd30-1d2854664c35","Type":"ContainerStarted","Data":"9a4c146c69aa536b6811aba6238da6f53ab02759b7d3c835714f7f5d9c953480"}
Jul 07 11:15:41.224551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.224524    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x" event={"ID":"c651db08-9dd6-4407-957c-088300bb56a6","Type":"ContainerStarted","Data":"9fe23a7891de5225c8481c885efd22fa7c383aa5053f16b53415396f7b040a99"}
Jul 07 11:15:41.224646 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.224598    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x" event={"ID":"c651db08-9dd6-4407-957c-088300bb56a6","Type":"ContainerStarted","Data":"3b6d7d52fff82af619f316ed6a41e03b8290a69d524b0158db75d43ab33f286c"}
Jul 07 11:15:41.224646 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.224642    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x" event={"ID":"c651db08-9dd6-4407-957c-088300bb56a6","Type":"ContainerStarted","Data":"357508c0386ffd70047c4f614cd375e04353009af2be5a8c48ec3e093a469f36"}
Jul 07 11:15:41.224740 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.224677    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x" event={"ID":"c651db08-9dd6-4407-957c-088300bb56a6","Type":"ContainerStarted","Data":"25a55098cd9ddf4778975d3c64f0e73e4290fd380e94932f556aa5990e052e24"}
Jul 07 11:15:41.226024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.226000    2582 generic.go:356] "Generic (PLEG): container finished" podID="5f055528432bc701b0b32a34b3d1cf9f" containerID="cdc6b03241eff579f6085f22ffd69a6bc0a410dcaf6a28a207c0bbe0a3dd9e83" exitCode=0
Jul 07 11:15:41.226127 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.226081    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal" event={"ID":"5f055528432bc701b0b32a34b3d1cf9f","Type":"ContainerDied","Data":"cdc6b03241eff579f6085f22ffd69a6bc0a410dcaf6a28a207c0bbe0a3dd9e83"}
Jul 07 11:15:41.227618 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.227593    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v" event={"ID":"d5a8106f-d2a4-4b29-bc33-dea166f5f249","Type":"ContainerStarted","Data":"67fcb2c97fbb0a4ffc97503f2f5dd9b4cdda60aff4db0466a670206bba7a304f"}
Jul 07 11:15:41.229210 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.229165    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-57cf9" event={"ID":"1113c816-17c6-479c-bb9e-36ea9b537bdb","Type":"ContainerStarted","Data":"81bb8bbe8740b61c74cf7e541e7f78aea3c540f0aab3a3d64585b4ba954c7313"}
Jul 07 11:15:41.230734 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.230698    2582 generic.go:356] "Generic (PLEG): container finished" podID="0266f5e8-632f-4c24-a508-abaa012b9483" containerID="fdb3a627cac797cb4e166b5fabe75be4dc146d0b755c790fe6fb03e6958e393f" exitCode=0
Jul 07 11:15:41.230843 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.230728    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m464b" event={"ID":"0266f5e8-632f-4c24-a508-abaa012b9483","Type":"ContainerDied","Data":"fdb3a627cac797cb4e166b5fabe75be4dc146d0b755c790fe6fb03e6958e393f"}
Jul 07 11:15:41.234586 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.234544    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cq9jg" podStartSLOduration=2.690866387 podStartE2EDuration="22.234529411s" totalImagesPullingTime="19.543663024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:41.234187464 +0000 UTC m=+22.772878876" watchObservedRunningTime="2026-07-07 11:15:41.234529411 +0000 UTC m=+22.773220823"
Jul 07 11:15:41.271730 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.271687    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ksbdk" podStartSLOduration=2.684850418 podStartE2EDuration="22.271674605s" totalImagesPullingTime="19.586824187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:41.2713658 +0000 UTC m=+22.810057213" watchObservedRunningTime="2026-07-07 11:15:41.271674605 +0000 UTC m=+22.810366014"
Jul 07 11:15:41.286705 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.286659    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-57cf9" podStartSLOduration=2.710799418 podStartE2EDuration="22.286642247s" totalImagesPullingTime="19.575842829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:41.286128709 +0000 UTC m=+22.824820116" watchObservedRunningTime="2026-07-07 11:15:41.286642247 +0000 UTC m=+22.825333664"
Jul 07 11:15:41.677708 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.677684    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 11:15:41.944035 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.943963    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:41.944644 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.944621    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:41.946822 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.946714    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T11:15:41.677702859Z","UUID":"8a34ccbe-a5c6-46d7-8c4b-e97dde3759e6","Handler":null,"Name":"","Endpoint":""}
Jul 07 11:15:41.948285 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.948258    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 11:15:41.948391 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:41.948302    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 11:15:42.019636 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:42.019612    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:42.019768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:42.019612    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:42.019768 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:42.019717    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:42.019862 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:42.019790    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:42.220607 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:42.220579    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:42.221183 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:42.221062    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cq9jg"
Jul 07 11:15:42.234627 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:42.234599    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m5jjb" event={"ID":"a451afbe-a804-458f-aee1-7340386ca641","Type":"ContainerStarted","Data":"5999360727a575f8dc4dc144a14357584b0b75b3f740206274045a85177ea1e6"}
Jul 07 11:15:42.236570 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:42.236536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal" event={"ID":"5f055528432bc701b0b32a34b3d1cf9f","Type":"ContainerStarted","Data":"b5f48a81cdb159ee4df4ff1766a4de77a3862c9c7410ac6ecb676ea0cb86ff4c"}
Jul 07 11:15:42.238719 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:42.238691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v" event={"ID":"d5a8106f-d2a4-4b29-bc33-dea166f5f249","Type":"ContainerStarted","Data":"bc122ba00bebf82da3919b8170873ccade62909b643487a5bd42be776c1f3bec"}
Jul 07 11:15:42.253119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:42.253073    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-180.ec2.internal" podStartSLOduration=23.253057234 podStartE2EDuration="23.253057234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:42.252942136 +0000 UTC m=+23.791633549" watchObservedRunningTime="2026-07-07 11:15:42.253057234 +0000 UTC m=+23.791748650"
Jul 07 11:15:42.267927 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:42.267884    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m5jjb" podStartSLOduration=4.579329134 podStartE2EDuration="23.26786994s" totalImagesPullingTime="18.688540806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:42.267604824 +0000 UTC m=+23.806296237" watchObservedRunningTime="2026-07-07 11:15:42.26786994 +0000 UTC m=+23.806561354"
Jul 07 11:15:43.020100 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:43.019863    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:43.020273 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:43.020243    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:43.243459 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:43.243426    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x" event={"ID":"c651db08-9dd6-4407-957c-088300bb56a6","Type":"ContainerStarted","Data":"71316ff27affd7346e6545e789ad2f6e9cc4e0ab63515105142aec2b3cf9e85a"}
Jul 07 11:15:43.245407 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:43.245380    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v" event={"ID":"d5a8106f-d2a4-4b29-bc33-dea166f5f249","Type":"ContainerStarted","Data":"59051f8adf067a6c164bc9b65f59cb18ef47f393b4f137ff68699e950ea3eda8"}
Jul 07 11:15:43.262989 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:43.262945    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdq6v" podStartSLOduration=1.9937817880000002 podStartE2EDuration="24.262931551s" totalImagesPullingTime="22.269149763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=3 observedRunningTime="2026-07-07 11:15:43.262334139 +0000 UTC m=+24.801025552" watchObservedRunningTime="2026-07-07 11:15:43.262931551 +0000 UTC m=+24.801622961"
Jul 07 11:15:44.019655 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:44.019622    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:44.019835 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:44.019622    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:44.019835 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:44.019748    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:44.019835 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:44.019787    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:45.019700 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:45.019413    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:45.020549 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:45.019756    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:45.253565 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:45.253535    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x" event={"ID":"c651db08-9dd6-4407-957c-088300bb56a6","Type":"ContainerStarted","Data":"1bbf5210361cdd4f2d2a68c9e4737b18ea9080a37770fa7f75d925a2cdf2f43b"}
Jul 07 11:15:45.253780 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:45.253751    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:45.253846 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:45.253816    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:45.270644 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:45.270546    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:45.286130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:45.286092    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x" podStartSLOduration=6.665264733 podStartE2EDuration="26.286081413s" totalImagesPullingTime="19.62081668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:45.285603861 +0000 UTC m=+26.824295272" watchObservedRunningTime="2026-07-07 11:15:45.286081413 +0000 UTC m=+26.824772825"
Jul 07 11:15:46.020094 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:46.019419    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:46.020094 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:46.019568    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:46.020094 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:46.019954    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:46.020094 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:46.020038    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:46.256005 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:46.255970    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:46.270791 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:46.270719    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:15:46.773612 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:46.773578    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cb8jb"]
Jul 07 11:15:46.773814 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:46.773726    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:46.773868 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:46.773827    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:46.774352 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:46.774318    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-524zs"]
Jul 07 11:15:46.774553 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:46.774425    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:46.774553 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:46.774517    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:46.775528 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:46.775491    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rfz9v"]
Jul 07 11:15:46.775641 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:46.775599    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:46.775786 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:46.775695    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:48.019367 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:48.019338    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:48.019804 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:48.019378    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:48.019804 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:48.019469    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:48.019804 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:48.019590    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:49.023221 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:49.022331    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:49.023221 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:49.022460    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:50.018978 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:50.018812    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:50.019107 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:50.018859    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:50.019157 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:50.019050    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:50.019157 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:50.019133    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:50.265262 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:50.265230    2582 generic.go:356] "Generic (PLEG): container finished" podID="0266f5e8-632f-4c24-a508-abaa012b9483" containerID="b25348b4d0a35013e9559cb0ef090d85a464d9156dd80fef061016884e64248f" exitCode=0
Jul 07 11:15:50.265593 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:50.265294    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m464b" event={"ID":"0266f5e8-632f-4c24-a508-abaa012b9483","Type":"ContainerDied","Data":"b25348b4d0a35013e9559cb0ef090d85a464d9156dd80fef061016884e64248f"}
Jul 07 11:15:51.019279 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:51.019252    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:51.019392 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:51.019368    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-524zs" podUID="4c33a0a8-99d2-4472-8aac-3b9cd92eaee2"
Jul 07 11:15:51.637333 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:51.637302    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:51.637641 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:51.637439    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:51.637641 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:51.637498    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs podName:0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca nodeName:}" failed. No retries permitted until 2026-07-07 11:16:23.637483408 +0000 UTC m=+65.176174802 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs") pod "network-metrics-daemon-rfz9v" (UID: "0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:15:51.738392 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:51.738366    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:51.738498 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:51.738480    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:51.738538 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:51.738528    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret podName:96412d1b-55ef-4558-88c3-4b83911151b5 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:23.738514182 +0000 UTC m=+65.277205572 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret") pod "global-pull-secret-syncer-cb8jb" (UID: "96412d1b-55ef-4558-88c3-4b83911151b5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:15:51.839124 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:51.839086    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldcr\" (UniqueName: \"kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr\") pod \"network-check-target-524zs\" (UID: \"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2\") " pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:51.839265 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:51.839249    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:15:51.839324 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:51.839271    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:15:51.839324 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:51.839280    2582 projected.go:196] Error preparing data for projected volume kube-api-access-4ldcr for pod openshift-network-diagnostics/network-check-target-524zs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:51.839393 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:51.839325    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr podName:4c33a0a8-99d2-4472-8aac-3b9cd92eaee2 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:23.839311955 +0000 UTC m=+65.378003348 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4ldcr" (UniqueName: "kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr") pod "network-check-target-524zs" (UID: "4c33a0a8-99d2-4472-8aac-3b9cd92eaee2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:15:52.019757 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.019700    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:52.019757 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.019735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:52.019889 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:52.019794    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rfz9v" podUID="0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca"
Jul 07 11:15:52.019889 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:52.019857    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cb8jb" podUID="96412d1b-55ef-4558-88c3-4b83911151b5"
Jul 07 11:15:52.271694 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.271624    2582 generic.go:356] "Generic (PLEG): container finished" podID="0266f5e8-632f-4c24-a508-abaa012b9483" containerID="66d04bb3089f169765db50f6bca9b9a4a9ef9f91ae6406276c6132d44551312f" exitCode=0
Jul 07 11:15:52.271694 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.271674    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m464b" event={"ID":"0266f5e8-632f-4c24-a508-abaa012b9483","Type":"ContainerDied","Data":"66d04bb3089f169765db50f6bca9b9a4a9ef9f91ae6406276c6132d44551312f"}
Jul 07 11:15:52.743728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.743704    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-180.ec2.internal" event="NodeReady"
Jul 07 11:15:52.744178 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.743808    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 11:15:52.777578 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.777543    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"]
Jul 07 11:15:52.789373 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.789352    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"]
Jul 07 11:15:52.789494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.789479    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:52.792593 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.792128    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 07 11:15:52.792593 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.792144    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 07 11:15:52.792593 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.792375    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:15:52.792795 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.792641    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 07 11:15:52.792886 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.792868    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-vk92s\""
Jul 07 11:15:52.796391 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.796370    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85d77785b6-h2klp"]
Jul 07 11:15:52.796517 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.796495    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:52.800547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.798992    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 07 11:15:52.800547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.799758    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 07 11:15:52.800547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.800051    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:15:52.800547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.800223    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 07 11:15:52.800547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.800248    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-pfj4l\""
Jul 07 11:15:52.816170 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.816153    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"]
Jul 07 11:15:52.816348 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.816330    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:52.819834 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.819812    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 11:15:52.822920 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.822902    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 11:15:52.822920 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.822919    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 11:15:52.823129 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.822927    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rbld5\""
Jul 07 11:15:52.826416 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.826397    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 11:15:52.829047 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.829030    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"]
Jul 07 11:15:52.829113 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.829056    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"]
Jul 07 11:15:52.829113 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.829068    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"]
Jul 07 11:15:52.829113 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.829085    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wxjtb"]
Jul 07 11:15:52.829239 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.829133    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:15:52.832358 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.832338    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:15:52.833070 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.833049    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 07 11:15:52.833157 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.833107    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 07 11:15:52.833440 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.833424    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9mmvf\""
Jul 07 11:15:52.843854 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.843835    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk"]
Jul 07 11:15:52.843968 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.843954    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:52.848097 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.848082    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 11:15:52.848364 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.848347    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 11:15:52.848451 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.848430    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgd79\""
Jul 07 11:15:52.859813 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.859798    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-rztt6"]
Jul 07 11:15:52.859921 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.859908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk"
Jul 07 11:15:52.863128 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.863113    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:15:52.863231 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.863159    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 07 11:15:52.864065 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.864033    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ng6jg\""
Jul 07 11:15:52.871014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.870993    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nbjlj"]
Jul 07 11:15:52.871943 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.871928    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:52.874797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.874779    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 07 11:15:52.874964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.874946    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 07 11:15:52.876823 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.876804    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 11:15:52.882438 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.882422    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mt9bn\""
Jul 07 11:15:52.888133 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.888114    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5c5dbcc7d5-crvzt"]
Jul 07 11:15:52.888267 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.888250    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:52.894782 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.894765    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 11:15:52.898160 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.898139    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 07 11:15:52.898160 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.898153    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 07 11:15:52.898332 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.898273    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-szgbx\""
Jul 07 11:15:52.898797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.898779    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:15:52.901110 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.901046    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 07 11:15:52.903596 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.903579    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 07 11:15:52.907408 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.907390    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"]
Jul 07 11:15:52.907518 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.907496    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:52.911414 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.911394    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-t4bj8\""
Jul 07 11:15:52.911507 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.911492    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 07 11:15:52.911607 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.911591    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 07 11:15:52.911662 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.911631    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 11:15:52.911662 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.911646    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 11:15:52.911757 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.911729    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 07 11:15:52.911876 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.911860    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 07 11:15:52.915404 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.915384    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 07 11:15:52.917798 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.917776    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kmx7n"]
Jul 07 11:15:52.917920 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.917902    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:52.920973 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.920956    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 07 11:15:52.921066 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.921001    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rnrbp\""
Jul 07 11:15:52.921472 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.921419    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 11:15:52.922260 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.922238    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 11:15:52.922692 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.922676    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 07 11:15:52.933338 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.933320    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk"]
Jul 07 11:15:52.933448 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.933434    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kmx7n"
Jul 07 11:15:52.936130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.936110    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 11:15:52.936236 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.936144    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 11:15:52.936236 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.936143    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 11:15:52.936236 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.936118    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hr2xm\""
Jul 07 11:15:52.944407 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.944380    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"]
Jul 07 11:15:52.944498 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.944484    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk"
Jul 07 11:15:52.947753 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.947726    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c98ff6c7-186a-42af-af1e-ba05d65dba1a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:52.947844 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.947761    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:52.947844 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.947805    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c98ff6c7-186a-42af-af1e-ba05d65dba1a-tmp\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:52.947844 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.947827    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-certificates\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:52.947844 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.947843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr4kx\" (UniqueName: \"kubernetes.io/projected/192da092-b9e0-4ca8-9f23-c752af83ca83-kube-api-access-zr4kx\") pod \"volume-data-source-validator-5d456fdbcd-75bnk\" (UID: \"192da092-b9e0-4ca8-9f23-c752af83ca83\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk"
Jul 07 11:15:52.948045 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.947866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8209861-6fc3-4980-8389-3cc58b1d3c40-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-cwqjb\" (UID: \"e8209861-6fc3-4980-8389-3cc58b1d3c40\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:52.948045 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.947889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c98ff6c7-186a-42af-af1e-ba05d65dba1a-serving-cert\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:52.948045 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.947919    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a58bd4ae-21c5-427d-a2e3-0791f820c597-ca-trust-extracted\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:52.948045 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.947971    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67qb4\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-kube-api-access-67qb4\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:52.948045 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948006    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq8qc\" (UniqueName: \"kubernetes.io/projected/cc5862d5-881b-445b-9abf-93f146315de8-kube-api-access-gq8qc\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:52.948313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948050    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxqj6\" (UniqueName: \"kubernetes.io/projected/40778e43-f44b-409e-8ed5-23f4971b8407-kube-api-access-cxqj6\") pod \"service-ca-operator-5d46d9ffd4-8xtcl\" (UID: \"40778e43-f44b-409e-8ed5-23f4971b8407\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:52.948313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948074    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c98ff6c7-186a-42af-af1e-ba05d65dba1a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:52.948313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948090    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:15:52.948313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948146    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc5862d5-881b-445b-9abf-93f146315de8-config-volume\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:52.948313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948180    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8gnt\" (UniqueName: \"kubernetes.io/projected/e8209861-6fc3-4980-8389-3cc58b1d3c40-kube-api-access-b8gnt\") pod \"kube-storage-version-migrator-operator-846bf668f7-cwqjb\" (UID: \"e8209861-6fc3-4980-8389-3cc58b1d3c40\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:52.948313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948235    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8209861-6fc3-4980-8389-3cc58b1d3c40-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-cwqjb\" (UID: \"e8209861-6fc3-4980-8389-3cc58b1d3c40\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:52.948313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948262    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmjrv\" (UniqueName: \"kubernetes.io/projected/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-kube-api-access-wmjrv\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:15:52.948313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948299    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-bound-sa-token\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:52.948728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948333    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r88tb\""
Jul 07 11:15:52.948728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-image-registry-private-configuration\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:52.948728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948385    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-trusted-ca\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:52.948728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948410    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5862d5-881b-445b-9abf-93f146315de8-tmp-dir\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:52.948728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948433    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl8fr\" (UniqueName: \"kubernetes.io/projected/c98ff6c7-186a-42af-af1e-ba05d65dba1a-kube-api-access-hl8fr\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:52.948728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948467    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c98ff6c7-186a-42af-af1e-ba05d65dba1a-snapshots\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:52.948728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948502    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40778e43-f44b-409e-8ed5-23f4971b8407-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-8xtcl\" (UID: \"40778e43-f44b-409e-8ed5-23f4971b8407\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:52.948728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948527    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:52.948728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948556    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/40778e43-f44b-409e-8ed5-23f4971b8407-config\") pod \"service-ca-operator-5d46d9ffd4-8xtcl\" (UID: \"40778e43-f44b-409e-8ed5-23f4971b8407\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:52.948728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948574    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-installation-pull-secrets\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:52.949215 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948775    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 11:15:52.949215 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.948967    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 11:15:52.956932 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.956915    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85d77785b6-h2klp"]
Jul 07 11:15:52.957014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.956945    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk"]
Jul 07 11:15:52.957014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.956981    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wxjtb"]
Jul 07 11:15:52.957014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.957001    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-rztt6"]
Jul 07 11:15:52.957014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.957013    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"]
Jul 07 11:15:52.957132 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.957018    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:15:52.957132 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.957025    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c5dbcc7d5-crvzt"]
Jul 07 11:15:52.957185 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.957147    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"]
Jul 07 11:15:52.957185 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.957174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nbjlj"]
Jul 07 11:15:52.957305 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.957234    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kmx7n"]
Jul 07 11:15:52.957305 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.957262    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk"]
Jul 07 11:15:52.959709 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.959689    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 07 11:15:52.959802 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.959693    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 07 11:15:52.959802 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:52.959723    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4mdc9\""
Jul 07 11:15:53.019586 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.019524    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:15:53.023573 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.022397    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42fbv\""
Jul 07 11:15:53.049026 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.048989    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8209861-6fc3-4980-8389-3cc58b1d3c40-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-cwqjb\" (UID: \"e8209861-6fc3-4980-8389-3cc58b1d3c40\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:53.049159 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049027    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c98ff6c7-186a-42af-af1e-ba05d65dba1a-serving-cert\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.049159 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049063    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:53.049335 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049182    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:15:53.049335 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049240    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a58bd4ae-21c5-427d-a2e3-0791f820c597-ca-trust-extracted\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.049335 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049266    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67qb4\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-kube-api-access-67qb4\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.049335 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049300    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8qc\" (UniqueName: \"kubernetes.io/projected/cc5862d5-881b-445b-9abf-93f146315de8-kube-api-access-gq8qc\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:53.049335 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049327    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-config\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.049546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049349    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-trusted-ca\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.049546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049373    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwlwb\" (UniqueName: \"kubernetes.io/projected/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-kube-api-access-gwlwb\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.049546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049405    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqj6\" (UniqueName: \"kubernetes.io/projected/40778e43-f44b-409e-8ed5-23f4971b8407-kube-api-access-cxqj6\") pod \"service-ca-operator-5d46d9ffd4-8xtcl\" (UID: \"40778e43-f44b-409e-8ed5-23f4971b8407\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:53.049546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c98ff6c7-186a-42af-af1e-ba05d65dba1a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.049546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049455    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-stats-auth\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.049546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049483    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:15:53.049546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.049546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049541    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d99c2e1c-47fd-42ad-ab43-6793dc71a73a-cert\") pod \"ingress-canary-kmx7n\" (UID: \"d99c2e1c-47fd-42ad-ab43-6793dc71a73a\") " pod="openshift-ingress-canary/ingress-canary-kmx7n"
Jul 07 11:15:53.049915 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049564    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a58bd4ae-21c5-427d-a2e3-0791f820c597-ca-trust-extracted\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.049959 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.049916    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:15:53.050021 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.050008    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls podName:ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:53.549991216 +0000 UTC m=+35.088682626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rf5tn" (UID: "ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56") : secret "samples-operator-tls" not found
Jul 07 11:15:53.050075 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050038    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc5862d5-881b-445b-9abf-93f146315de8-config-volume\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:53.050134 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.049567    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc5862d5-881b-445b-9abf-93f146315de8-config-volume\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:53.050187 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8gnt\" (UniqueName: \"kubernetes.io/projected/e8209861-6fc3-4980-8389-3cc58b1d3c40-kube-api-access-b8gnt\") pod \"kube-storage-version-migrator-operator-846bf668f7-cwqjb\" (UID: \"e8209861-6fc3-4980-8389-3cc58b1d3c40\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:53.050187 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050172    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8209861-6fc3-4980-8389-3cc58b1d3c40-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-cwqjb\" (UID: \"e8209861-6fc3-4980-8389-3cc58b1d3c40\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:53.050311 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050219    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmjrv\" (UniqueName: \"kubernetes.io/projected/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-kube-api-access-wmjrv\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:15:53.050311 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050249    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-serving-cert\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.050311 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050277    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9b40451d-1272-4a5d-a8be-fa8a20b6b565-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:15:53.050434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050309    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-bound-sa-token\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.050434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050338    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-image-registry-private-configuration\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.050434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050373    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnxqn\" (UniqueName: \"kubernetes.io/projected/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-kube-api-access-mnxqn\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:53.050434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050434    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-default-certificate\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.050613 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050480    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.050613 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:53.050613 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050554    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvqdx\" (UniqueName: \"kubernetes.io/projected/81328b2b-d34d-4d93-af43-4d80f238f9f1-kube-api-access-hvqdx\") pod \"network-check-source-989f8b8dc-d7xvk\" (UID: \"81328b2b-d34d-4d93-af43-4d80f238f9f1\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk"
Jul 07 11:15:53.050613 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-trusted-ca\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.050787 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050645    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c98ff6c7-186a-42af-af1e-ba05d65dba1a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.050787 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050655    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8209861-6fc3-4980-8389-3cc58b1d3c40-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-cwqjb\" (UID: \"e8209861-6fc3-4980-8389-3cc58b1d3c40\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:53.050787 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050691    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5862d5-881b-445b-9abf-93f146315de8-tmp-dir\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:53.050913 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050795    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl8fr\" (UniqueName: \"kubernetes.io/projected/c98ff6c7-186a-42af-af1e-ba05d65dba1a-kube-api-access-hl8fr\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.050955 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.050922    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c98ff6c7-186a-42af-af1e-ba05d65dba1a-snapshots\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.051055 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051019    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5862d5-881b-445b-9abf-93f146315de8-tmp-dir\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:53.051121 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40778e43-f44b-409e-8ed5-23f4971b8407-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-8xtcl\" (UID: \"40778e43-f44b-409e-8ed5-23f4971b8407\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051187    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051252    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/40778e43-f44b-409e-8ed5-23f4971b8407-config\") pod \"service-ca-operator-5d46d9ffd4-8xtcl\" (UID: \"40778e43-f44b-409e-8ed5-23f4971b8407\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.051275    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051284    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-installation-pull-secrets\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051313    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c98ff6c7-186a-42af-af1e-ba05d65dba1a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.051333    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls podName:cc5862d5-881b-445b-9abf-93f146315de8 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:53.551317341 +0000 UTC m=+35.090008744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls") pod "dns-default-wxjtb" (UID: "cc5862d5-881b-445b-9abf-93f146315de8") : secret "dns-default-metrics-tls" not found
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051360    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051388    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zztdh\" (UniqueName: \"kubernetes.io/projected/1de016bd-1463-4d64-ad1b-09e17dce5d18-kube-api-access-zztdh\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c98ff6c7-186a-42af-af1e-ba05d65dba1a-tmp\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051440    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlnw8\" (UniqueName: \"kubernetes.io/projected/d99c2e1c-47fd-42ad-ab43-6793dc71a73a-kube-api-access-tlnw8\") pod \"ingress-canary-kmx7n\" (UID: \"d99c2e1c-47fd-42ad-ab43-6793dc71a73a\") " pod="openshift-ingress-canary/ingress-canary-kmx7n"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051467    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-certificates\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051491    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c98ff6c7-186a-42af-af1e-ba05d65dba1a-snapshots\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051692    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-trusted-ca\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr4kx\" (UniqueName: \"kubernetes.io/projected/192da092-b9e0-4ca8-9f23-c752af83ca83-kube-api-access-zr4kx\") pod \"volume-data-source-validator-5d456fdbcd-75bnk\" (UID: \"192da092-b9e0-4ca8-9f23-c752af83ca83\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051755    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c98ff6c7-186a-42af-af1e-ba05d65dba1a-tmp\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.051821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051772    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c98ff6c7-186a-42af-af1e-ba05d65dba1a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.052764 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.051957    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/40778e43-f44b-409e-8ed5-23f4971b8407-config\") pod \"service-ca-operator-5d46d9ffd4-8xtcl\" (UID: \"40778e43-f44b-409e-8ed5-23f4971b8407\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:53.052764 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.051980    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:15:53.052764 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.051997    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d77785b6-h2klp: secret "image-registry-tls" not found
Jul 07 11:15:53.052764 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.052056    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls podName:a58bd4ae-21c5-427d-a2e3-0791f820c597 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:53.55203923 +0000 UTC m=+35.090730637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls") pod "image-registry-85d77785b6-h2klp" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597") : secret "image-registry-tls" not found
Jul 07 11:15:53.052764 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.052226    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-certificates\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.054226 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.054187    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40778e43-f44b-409e-8ed5-23f4971b8407-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-8xtcl\" (UID: \"40778e43-f44b-409e-8ed5-23f4971b8407\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:53.054386 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.054283    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c98ff6c7-186a-42af-af1e-ba05d65dba1a-serving-cert\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.054386 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.054333    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-image-registry-private-configuration\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.054386 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.054334    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-installation-pull-secrets\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.054613 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.054594    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8209861-6fc3-4980-8389-3cc58b1d3c40-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-cwqjb\" (UID: \"e8209861-6fc3-4980-8389-3cc58b1d3c40\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:53.059965 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.059915    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8gnt\" (UniqueName: \"kubernetes.io/projected/e8209861-6fc3-4980-8389-3cc58b1d3c40-kube-api-access-b8gnt\") pod \"kube-storage-version-migrator-operator-846bf668f7-cwqjb\" (UID: \"e8209861-6fc3-4980-8389-3cc58b1d3c40\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:53.060864 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.060810    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67qb4\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-kube-api-access-67qb4\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.060864 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.060817    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq8qc\" (UniqueName: \"kubernetes.io/projected/cc5862d5-881b-445b-9abf-93f146315de8-kube-api-access-gq8qc\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:53.061243 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.061220    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxqj6\" (UniqueName: \"kubernetes.io/projected/40778e43-f44b-409e-8ed5-23f4971b8407-kube-api-access-cxqj6\") pod \"service-ca-operator-5d46d9ffd4-8xtcl\" (UID: \"40778e43-f44b-409e-8ed5-23f4971b8407\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:53.061579 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.061536    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr4kx\" (UniqueName: \"kubernetes.io/projected/192da092-b9e0-4ca8-9f23-c752af83ca83-kube-api-access-zr4kx\") pod \"volume-data-source-validator-5d456fdbcd-75bnk\" (UID: \"192da092-b9e0-4ca8-9f23-c752af83ca83\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk"
Jul 07 11:15:53.061664 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.061574    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl8fr\" (UniqueName: \"kubernetes.io/projected/c98ff6c7-186a-42af-af1e-ba05d65dba1a-kube-api-access-hl8fr\") pod \"insights-operator-5c847dcfc4-rztt6\" (UID: \"c98ff6c7-186a-42af-af1e-ba05d65dba1a\") " pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.062998 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.062944    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-bound-sa-token\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.063962 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.063942    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmjrv\" (UniqueName: \"kubernetes.io/projected/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-kube-api-access-wmjrv\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:15:53.103702 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.103679    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"
Jul 07 11:15:53.110815 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.110792    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"
Jul 07 11:15:53.152165 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zztdh\" (UniqueName: \"kubernetes.io/projected/1de016bd-1463-4d64-ad1b-09e17dce5d18-kube-api-access-zztdh\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.152271 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152169    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlnw8\" (UniqueName: \"kubernetes.io/projected/d99c2e1c-47fd-42ad-ab43-6793dc71a73a-kube-api-access-tlnw8\") pod \"ingress-canary-kmx7n\" (UID: \"d99c2e1c-47fd-42ad-ab43-6793dc71a73a\") " pod="openshift-ingress-canary/ingress-canary-kmx7n"
Jul 07 11:15:53.152271 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152212    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:53.152271 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:15:53.152434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152288    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-config\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.152434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152312    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-trusted-ca\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.152434 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.152320    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:15:53.152434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152331    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwlwb\" (UniqueName: \"kubernetes.io/projected/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-kube-api-access-gwlwb\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.152434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152351    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-stats-auth\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.152434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152385    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.152434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d99c2e1c-47fd-42ad-ab43-6793dc71a73a-cert\") pod \"ingress-canary-kmx7n\" (UID: \"d99c2e1c-47fd-42ad-ab43-6793dc71a73a\") " pod="openshift-ingress-canary/ingress-canary-kmx7n"
Jul 07 11:15:53.152847 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152457    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-serving-cert\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.152847 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152480    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9b40451d-1272-4a5d-a8be-fa8a20b6b565-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:15:53.152847 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnxqn\" (UniqueName: \"kubernetes.io/projected/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-kube-api-access-mnxqn\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:53.152847 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.152551    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:15:53.152847 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.152614    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:53.652595748 +0000 UTC m=+35.191287139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : secret "router-metrics-certs-default" not found
Jul 07 11:15:53.153119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152553    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-default-certificate\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.153119 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.152928    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert podName:9b40451d-1272-4a5d-a8be-fa8a20b6b565 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:53.652906071 +0000 UTC m=+35.191597461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wshqb" (UID: "9b40451d-1272-4a5d-a8be-fa8a20b6b565") : secret "networking-console-plugin-cert" not found
Jul 07 11:15:53.153119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152930    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:53.153119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.152969    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.153119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.153014    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:53.153119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.153029    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-config\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.153119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.153061    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvqdx\" (UniqueName: \"kubernetes.io/projected/81328b2b-d34d-4d93-af43-4d80f238f9f1-kube-api-access-hvqdx\") pod \"network-check-source-989f8b8dc-d7xvk\" (UID: \"81328b2b-d34d-4d93-af43-4d80f238f9f1\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk"
Jul 07 11:15:53.153494 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.153135    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:53.653119598 +0000 UTC m=+35.191810996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : configmap references non-existent config key: service-ca.crt
Jul 07 11:15:53.153494 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.153322    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:15:53.153494 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.153369    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls podName:1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:53.653355566 +0000 UTC m=+35.192046959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-k46bt" (UID: "1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:15:53.153778 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.153760    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-trusted-ca\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.153952 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.153918    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9b40451d-1272-4a5d-a8be-fa8a20b6b565-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:15:53.155257 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.155234    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-stats-auth\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.155531 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.155511    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-default-certificate\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.155531 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.155523    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-serving-cert\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.155615 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.155603    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d99c2e1c-47fd-42ad-ab43-6793dc71a73a-cert\") pod \"ingress-canary-kmx7n\" (UID: \"d99c2e1c-47fd-42ad-ab43-6793dc71a73a\") " pod="openshift-ingress-canary/ingress-canary-kmx7n"
Jul 07 11:15:53.161626 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.161581    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zztdh\" (UniqueName: \"kubernetes.io/projected/1de016bd-1463-4d64-ad1b-09e17dce5d18-kube-api-access-zztdh\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.162227 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.162167    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlnw8\" (UniqueName: \"kubernetes.io/projected/d99c2e1c-47fd-42ad-ab43-6793dc71a73a-kube-api-access-tlnw8\") pod \"ingress-canary-kmx7n\" (UID: \"d99c2e1c-47fd-42ad-ab43-6793dc71a73a\") " pod="openshift-ingress-canary/ingress-canary-kmx7n"
Jul 07 11:15:53.162319 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.162248    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwlwb\" (UniqueName: \"kubernetes.io/projected/228f0fc4-cb8d-4540-97cb-3f794bbbeb63-kube-api-access-gwlwb\") pod \"console-operator-68b757fb67-nbjlj\" (UID: \"228f0fc4-cb8d-4540-97cb-3f794bbbeb63\") " pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.162601 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.162563    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvqdx\" (UniqueName: \"kubernetes.io/projected/81328b2b-d34d-4d93-af43-4d80f238f9f1-kube-api-access-hvqdx\") pod \"network-check-source-989f8b8dc-d7xvk\" (UID: \"81328b2b-d34d-4d93-af43-4d80f238f9f1\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk"
Jul 07 11:15:53.162838 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.162797    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnxqn\" (UniqueName: \"kubernetes.io/projected/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-kube-api-access-mnxqn\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:53.170184 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.170155    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk"
Jul 07 11:15:53.182994 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.182978    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-rztt6"
Jul 07 11:15:53.199472 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.199414    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:15:53.247796 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.247413    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kmx7n"
Jul 07 11:15:53.253502 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.253433    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk"
Jul 07 11:15:53.359737 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.359699    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl"]
Jul 07 11:15:53.365804 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.364401    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb"]
Jul 07 11:15:53.366778 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.366756    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk"]
Jul 07 11:15:53.394383 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.394355    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-rztt6"]
Jul 07 11:15:53.423931 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.423855    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nbjlj"]
Jul 07 11:15:53.433763 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:53.433712    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8209861_6fc3_4980_8389_3cc58b1d3c40.slice/crio-d73ca43621fd7d38b56f46f68ea82d972d828b760acd78c933657269d5fe5080 WatchSource:0}: Error finding container d73ca43621fd7d38b56f46f68ea82d972d828b760acd78c933657269d5fe5080: Status 404 returned error can't find the container with id d73ca43621fd7d38b56f46f68ea82d972d828b760acd78c933657269d5fe5080
Jul 07 11:15:53.434928 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:53.434529    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40778e43_f44b_409e_8ed5_23f4971b8407.slice/crio-03632e3004828f69834e48a6b90d5755e4f083dac5e872b3b2daf64039b1da87 WatchSource:0}: Error finding container 03632e3004828f69834e48a6b90d5755e4f083dac5e872b3b2daf64039b1da87: Status 404 returned error can't find the container with id 03632e3004828f69834e48a6b90d5755e4f083dac5e872b3b2daf64039b1da87
Jul 07 11:15:53.435641 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.435053    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kmx7n"]
Jul 07 11:15:53.435928 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:53.435793    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192da092_b9e0_4ca8_9f23_c752af83ca83.slice/crio-c971459771d394d61c28a2024e45187b20b07ec038b49461ab66caf08edc54f4 WatchSource:0}: Error finding container c971459771d394d61c28a2024e45187b20b07ec038b49461ab66caf08edc54f4: Status 404 returned error can't find the container with id c971459771d394d61c28a2024e45187b20b07ec038b49461ab66caf08edc54f4
Jul 07 11:15:53.436620 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:53.436124    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98ff6c7_186a_42af_af1e_ba05d65dba1a.slice/crio-0b8986b0f92b101f8ecc2f78b2cc208ed9303c9004a62dd73b694e974fedb008 WatchSource:0}: Error finding container 0b8986b0f92b101f8ecc2f78b2cc208ed9303c9004a62dd73b694e974fedb008: Status 404 returned error can't find the container with id 0b8986b0f92b101f8ecc2f78b2cc208ed9303c9004a62dd73b694e974fedb008
Jul 07 11:15:53.440905 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:53.440826    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod228f0fc4_cb8d_4540_97cb_3f794bbbeb63.slice/crio-9e24f3c762b457b4a67426ac6dea218d7871e307b8a8b08a4a9a2c71cc31bb1e WatchSource:0}: Error finding container 9e24f3c762b457b4a67426ac6dea218d7871e307b8a8b08a4a9a2c71cc31bb1e: Status 404 returned error can't find the container with id 9e24f3c762b457b4a67426ac6dea218d7871e307b8a8b08a4a9a2c71cc31bb1e
Jul 07 11:15:53.441115 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:53.441071    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99c2e1c_47fd_42ad_ab43_6793dc71a73a.slice/crio-13910ac432eb14ffb159bab049f5c1c001c62e4b15be69477bb47e01565ffa20 WatchSource:0}: Error finding container 13910ac432eb14ffb159bab049f5c1c001c62e4b15be69477bb47e01565ffa20: Status 404 returned error can't find the container with id 13910ac432eb14ffb159bab049f5c1c001c62e4b15be69477bb47e01565ffa20
Jul 07 11:15:53.461584 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.461563    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk"]
Jul 07 11:15:53.463475 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:15:53.463456    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81328b2b_d34d_4d93_af43_4d80f238f9f1.slice/crio-fd203c5f8f95d61f4c4d32b5ba03d6fd69b390bcc9ef469ced41b385d984391c WatchSource:0}: Error finding container fd203c5f8f95d61f4c4d32b5ba03d6fd69b390bcc9ef469ced41b385d984391c: Status 404 returned error can't find the container with id fd203c5f8f95d61f4c4d32b5ba03d6fd69b390bcc9ef469ced41b385d984391c
Jul 07 11:15:53.556725 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.556704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:15:53.556860 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.556842    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:15:53.556918 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.556909    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls podName:ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:54.556892351 +0000 UTC m=+36.095583756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rf5tn" (UID: "ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56") : secret "samples-operator-tls" not found
Jul 07 11:15:53.556962 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.556928    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:53.556992 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.556971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:53.557072 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.557060    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:15:53.557109 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.557075    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d77785b6-h2klp: secret "image-registry-tls" not found
Jul 07 11:15:53.557109 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.557087    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:15:53.557167 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.557108    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls podName:a58bd4ae-21c5-427d-a2e3-0791f820c597 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:54.557097419 +0000 UTC m=+36.095788813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls") pod "image-registry-85d77785b6-h2klp" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597") : secret "image-registry-tls" not found
Jul 07 11:15:53.557167 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.557125    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls podName:cc5862d5-881b-445b-9abf-93f146315de8 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:54.557113519 +0000 UTC m=+36.095804927 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls") pod "dns-default-wxjtb" (UID: "cc5862d5-881b-445b-9abf-93f146315de8") : secret "dns-default-metrics-tls" not found
Jul 07 11:15:53.657715 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.657688    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:15:53.657817 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.657742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.657869 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.657812    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:53.657869 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:53.657838    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:53.657869 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.657842    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:15:53.657995 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.657894    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:15:53.657995 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.657903    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert podName:9b40451d-1272-4a5d-a8be-fa8a20b6b565 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:54.657885026 +0000 UTC m=+36.196576439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wshqb" (UID: "9b40451d-1272-4a5d-a8be-fa8a20b6b565") : secret "networking-console-plugin-cert" not found
Jul 07 11:15:53.657995 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.657930    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:15:53.657995 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.657973    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:54.657944817 +0000 UTC m=+36.196636220 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : configmap references non-existent config key: service-ca.crt
Jul 07 11:15:53.658293 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.657999    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:54.657988847 +0000 UTC m=+36.196680241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : secret "router-metrics-certs-default" not found
Jul 07 11:15:53.658293 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:53.658041    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls podName:1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:54.658031372 +0000 UTC m=+36.196722766 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-k46bt" (UID: "1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:15:54.019601 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.019568    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:15:54.019601 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.019601    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:15:54.022798 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.022776    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 11:15:54.022950 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.022819    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 11:15:54.022950 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.022851    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89jkb\""
Jul 07 11:15:54.287254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.287167    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rztt6" event={"ID":"c98ff6c7-186a-42af-af1e-ba05d65dba1a","Type":"ContainerStarted","Data":"0b8986b0f92b101f8ecc2f78b2cc208ed9303c9004a62dd73b694e974fedb008"}
Jul 07 11:15:54.290618 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.290586    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl" event={"ID":"40778e43-f44b-409e-8ed5-23f4971b8407","Type":"ContainerStarted","Data":"03632e3004828f69834e48a6b90d5755e4f083dac5e872b3b2daf64039b1da87"}
Jul 07 11:15:54.292361 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.292335    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kmx7n" event={"ID":"d99c2e1c-47fd-42ad-ab43-6793dc71a73a","Type":"ContainerStarted","Data":"13910ac432eb14ffb159bab049f5c1c001c62e4b15be69477bb47e01565ffa20"}
Jul 07 11:15:54.295357 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.295288    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" event={"ID":"228f0fc4-cb8d-4540-97cb-3f794bbbeb63","Type":"ContainerStarted","Data":"9e24f3c762b457b4a67426ac6dea218d7871e307b8a8b08a4a9a2c71cc31bb1e"}
Jul 07 11:15:54.297000 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.296974    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk" event={"ID":"192da092-b9e0-4ca8-9f23-c752af83ca83","Type":"ContainerStarted","Data":"c971459771d394d61c28a2024e45187b20b07ec038b49461ab66caf08edc54f4"}
Jul 07 11:15:54.298988 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.298943    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb" event={"ID":"e8209861-6fc3-4980-8389-3cc58b1d3c40","Type":"ContainerStarted","Data":"d73ca43621fd7d38b56f46f68ea82d972d828b760acd78c933657269d5fe5080"}
Jul 07 11:15:54.303550 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.302842    2582 generic.go:356] "Generic (PLEG): container finished" podID="0266f5e8-632f-4c24-a508-abaa012b9483" containerID="753c15d760e4eac88d90b9d15cf08a57602e2b68d7973974df52b0b44ad64c84" exitCode=0
Jul 07 11:15:54.303550 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.302906    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m464b" event={"ID":"0266f5e8-632f-4c24-a508-abaa012b9483","Type":"ContainerDied","Data":"753c15d760e4eac88d90b9d15cf08a57602e2b68d7973974df52b0b44ad64c84"}
Jul 07 11:15:54.314010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.309371    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk" event={"ID":"81328b2b-d34d-4d93-af43-4d80f238f9f1","Type":"ContainerStarted","Data":"fd203c5f8f95d61f4c4d32b5ba03d6fd69b390bcc9ef469ced41b385d984391c"}
Jul 07 11:15:54.566880 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.566799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:54.566880 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.566872    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:54.567087 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.566942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:15:54.567145 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.567088    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:15:54.567219 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.567149    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls podName:ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:56.567131173 +0000 UTC m=+38.105822566 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rf5tn" (UID: "ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56") : secret "samples-operator-tls" not found
Jul 07 11:15:54.567485 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.567309    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:15:54.567485 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.567325    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d77785b6-h2klp: secret "image-registry-tls" not found
Jul 07 11:15:54.567485 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.567365    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls podName:a58bd4ae-21c5-427d-a2e3-0791f820c597 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:56.567351557 +0000 UTC m=+38.106042951 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls") pod "image-registry-85d77785b6-h2klp" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597") : secret "image-registry-tls" not found
Jul 07 11:15:54.567485 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.567425    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:15:54.567485 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.567454    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls podName:cc5862d5-881b-445b-9abf-93f146315de8 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:56.567444816 +0000 UTC m=+38.106136210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls") pod "dns-default-wxjtb" (UID: "cc5862d5-881b-445b-9abf-93f146315de8") : secret "dns-default-metrics-tls" not found
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.668379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.668457    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.668530    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:54.668559    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.668763    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.668824    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls podName:1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:56.668806032 +0000 UTC m=+38.207497436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-k46bt" (UID: "1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.669245    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.669297    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert podName:9b40451d-1272-4a5d-a8be-fa8a20b6b565 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:56.669280989 +0000 UTC m=+38.207972392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wshqb" (UID: "9b40451d-1272-4a5d-a8be-fa8a20b6b565") : secret "networking-console-plugin-cert" not found
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.669367    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.669406    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:56.669394017 +0000 UTC m=+38.208085409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : secret "router-metrics-certs-default" not found
Jul 07 11:15:54.669510 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:54.669475    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:15:56.66946332 +0000 UTC m=+38.208154712 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : configmap references non-existent config key: service-ca.crt
Jul 07 11:15:56.458066 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.458028    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:15:56.589731 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.589655    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:15:56.589885 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.589726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:15:56.589885 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.589804    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:15:56.590002 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.589986    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:15:56.590057 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.590047    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls podName:cc5862d5-881b-445b-9abf-93f146315de8 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.590028372 +0000 UTC m=+42.128719765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls") pod "dns-default-wxjtb" (UID: "cc5862d5-881b-445b-9abf-93f146315de8") : secret "dns-default-metrics-tls" not found
Jul 07 11:15:56.590473 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.590455    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:15:56.590555 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.590476    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d77785b6-h2klp: secret "image-registry-tls" not found
Jul 07 11:15:56.590555 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.590518    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls podName:a58bd4ae-21c5-427d-a2e3-0791f820c597 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.59050496 +0000 UTC m=+42.129196351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls") pod "image-registry-85d77785b6-h2klp" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597") : secret "image-registry-tls" not found
Jul 07 11:15:56.590555 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.590553    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:15:56.590694 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.590585    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls podName:ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.590574185 +0000 UTC m=+42.129265576 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rf5tn" (UID: "ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56") : secret "samples-operator-tls" not found
Jul 07 11:15:56.593901 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.593878    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:15:56.690852 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.690778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:56.690852 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.690826    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:15:56.691093 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.690945    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:15:56.691093 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.691005    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.690982113 +0000 UTC m=+42.229673504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : configmap references non-existent config key: service-ca.crt
Jul 07 11:15:56.691093 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.691040    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:15:56.691093 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.691055    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:15:56.691093 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.691062    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:15:56.691394 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.691104    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls podName:1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.691087087 +0000 UTC m=+42.229778493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-k46bt" (UID: "1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:15:56.691394 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.691114    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:15:56.691394 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.691129    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert podName:9b40451d-1272-4a5d-a8be-fa8a20b6b565 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.691117197 +0000 UTC m=+42.229808594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wshqb" (UID: "9b40451d-1272-4a5d-a8be-fa8a20b6b565") : secret "networking-console-plugin-cert" not found
Jul 07 11:15:56.691394 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:15:56.691159    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.691145804 +0000 UTC m=+42.229837212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : secret "router-metrics-certs-default" not found
Jul 07 11:15:56.693173 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.693133    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:15:56.793955 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:56.793870    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:15:57.382177 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:57.382136    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:15:57.441260 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:57.441222    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:15:57.786444 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:57.786403    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:15:57.889511 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:57.889443    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:15:58.063962 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:58.063856    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:15:58.118212 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:15:58.118150    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:00.632756 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:00.632713    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:16:00.633237 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.632880    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:16:00.633237 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:00.632913    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:16:00.633237 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.632953    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls podName:ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:08.632931085 +0000 UTC m=+50.171622478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rf5tn" (UID: "ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56") : secret "samples-operator-tls" not found
Jul 07 11:16:00.633237 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:00.632999    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:16:00.633237 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.633010    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:16:00.633237 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.633059    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls podName:cc5862d5-881b-445b-9abf-93f146315de8 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:08.633046402 +0000 UTC m=+50.171737793 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls") pod "dns-default-wxjtb" (UID: "cc5862d5-881b-445b-9abf-93f146315de8") : secret "dns-default-metrics-tls" not found
Jul 07 11:16:00.633237 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.633098    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:16:00.633237 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.633112    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d77785b6-h2klp: secret "image-registry-tls" not found
Jul 07 11:16:00.633237 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.633147    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls podName:a58bd4ae-21c5-427d-a2e3-0791f820c597 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:08.633136464 +0000 UTC m=+50.171827868 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls") pod "image-registry-85d77785b6-h2klp" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597") : secret "image-registry-tls" not found
Jul 07 11:16:00.699434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:00.699335    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:00.733543 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:00.733516    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:00.733680 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:00.733549    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:16:00.733680 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.733668    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:16:00.733777 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.733670    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:08.733652948 +0000 UTC m=+50.272344355 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : configmap references non-existent config key: service-ca.crt
Jul 07 11:16:00.733832 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:00.733773    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:16:00.733872 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.733843    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls podName:1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:08.733815843 +0000 UTC m=+50.272507235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-k46bt" (UID: "1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:16:00.733872 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.733865    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:16:00.733967 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:00.733876    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:00.733967 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.733903    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert podName:9b40451d-1272-4a5d-a8be-fa8a20b6b565 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:08.73389134 +0000 UTC m=+50.272582730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wshqb" (UID: "9b40451d-1272-4a5d-a8be-fa8a20b6b565") : secret "networking-console-plugin-cert" not found
Jul 07 11:16:00.733967 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.733957    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:16:00.734098 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:00.734017    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:08.734005367 +0000 UTC m=+50.272696756 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : secret "router-metrics-certs-default" not found
Jul 07 11:16:00.800692 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:00.800661    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:03.338492 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:03.338220    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" event={"ID":"228f0fc4-cb8d-4540-97cb-3f794bbbeb63","Type":"ContainerStarted","Data":"0bf72144da08997edb2ba3a6ba0188557eebc9631ac368f17ef797db967ee0ba"}
Jul 07 11:16:03.340132 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:03.340108    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk" event={"ID":"192da092-b9e0-4ca8-9f23-c752af83ca83","Type":"ContainerStarted","Data":"3a0dd4ee06d0663427428ec11d24460731100ad0cc91d4bad16b35280b8e833a"}
Jul 07 11:16:03.341798 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:03.341769    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rztt6" event={"ID":"c98ff6c7-186a-42af-af1e-ba05d65dba1a","Type":"ContainerStarted","Data":"0be06590e2a3d61f26a25442ce0e39ce23cef65b35ba4f8fe2e1d60cb9b4428c"}
Jul 07 11:16:03.380099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:03.379999    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:16:03.381036 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:03.381014    2582 patch_prober.go:28] interesting pod/console-operator-68b757fb67-nbjlj container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.16:8443/readyz\": dial tcp 10.132.0.16:8443: connect: connection refused" start-of-body=
Jul 07 11:16:03.381113 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:03.381060    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" podUID="228f0fc4-cb8d-4540-97cb-3f794bbbeb63" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.16:8443/readyz\": dial tcp 10.132.0.16:8443: connect: connection refused"
Jul 07 11:16:03.432804 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:03.430756    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-75bnk" podStartSLOduration=12.917569228 podStartE2EDuration="20.430738029s" totalImagesPullingTime="7.513168801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:03.393071658 +0000 UTC m=+44.931763071" watchObservedRunningTime="2026-07-07 11:16:03.430738029 +0000 UTC m=+44.969429442"
Jul 07 11:16:03.460094 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:03.459085    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" podStartSLOduration=10.751812667 podStartE2EDuration="20.459062966s" totalImagesPullingTime="9.707250299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:03.433109497 +0000 UTC m=+44.971800910" watchObservedRunningTime="2026-07-07 11:16:03.459062966 +0000 UTC m=+44.997754380"
Jul 07 11:16:03.562025 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:03.561974    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:03.658118 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:03.658075    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:04.347766 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.347294    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kmx7n" event={"ID":"d99c2e1c-47fd-42ad-ab43-6793dc71a73a","Type":"ContainerStarted","Data":"97fef74a9e9bc7fdade7e05c18bc28ed9c19c3cf57b866b302e22ddd8fc0a229"}
Jul 07 11:16:04.349810 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.349788    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/0.log"
Jul 07 11:16:04.349941 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.349831    2582 generic.go:356] "Generic (PLEG): container finished" podID="228f0fc4-cb8d-4540-97cb-3f794bbbeb63" containerID="0bf72144da08997edb2ba3a6ba0188557eebc9631ac368f17ef797db967ee0ba" exitCode=255
Jul 07 11:16:04.349941 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.349914    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" event={"ID":"228f0fc4-cb8d-4540-97cb-3f794bbbeb63","Type":"ContainerDied","Data":"0bf72144da08997edb2ba3a6ba0188557eebc9631ac368f17ef797db967ee0ba"}
Jul 07 11:16:04.350467 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.350451    2582 scope.go:117] "RemoveContainer" containerID="0bf72144da08997edb2ba3a6ba0188557eebc9631ac368f17ef797db967ee0ba"
Jul 07 11:16:04.352108 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.351659    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb" event={"ID":"e8209861-6fc3-4980-8389-3cc58b1d3c40","Type":"ContainerStarted","Data":"4b881dd6d559e1ef5f72d51d7fafce391fcef6f55bc1b611d724af8defde226d"}
Jul 07 11:16:04.355829 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.355558    2582 generic.go:356] "Generic (PLEG): container finished" podID="0266f5e8-632f-4c24-a508-abaa012b9483" containerID="3e8585f9679264d7da2689a5bcad01f8cf4a629dc3cc1f0ba867029671580722" exitCode=0
Jul 07 11:16:04.355829 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.355627    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m464b" event={"ID":"0266f5e8-632f-4c24-a508-abaa012b9483","Type":"ContainerDied","Data":"3e8585f9679264d7da2689a5bcad01f8cf4a629dc3cc1f0ba867029671580722"}
Jul 07 11:16:04.360064 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.359977    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk" event={"ID":"81328b2b-d34d-4d93-af43-4d80f238f9f1","Type":"ContainerStarted","Data":"b31f5e98d5bcdfec9f74f289a11b88159ec22b9a2b1931309871bd3b28d6f51b"}
Jul 07 11:16:04.362582 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.362557    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl" event={"ID":"40778e43-f44b-409e-8ed5-23f4971b8407","Type":"ContainerStarted","Data":"d26394751a76f3ca37ed47f7880bfda90536266d5bb346023234a4fac50763be"}
Jul 07 11:16:04.389262 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.388999    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-rztt6" podStartSLOduration=11.681432177 podStartE2EDuration="21.388977686s" totalImagesPullingTime="9.707545509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:03.459657778 +0000 UTC m=+44.998349194" watchObservedRunningTime="2026-07-07 11:16:04.388977686 +0000 UTC m=+45.927669101"
Jul 07 11:16:04.389602 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.389572    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kmx7n" podStartSLOduration=2.681969117 podStartE2EDuration="12.389561312s" totalImagesPullingTime="9.707592195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:04.388824836 +0000 UTC m=+45.927516267" watchObservedRunningTime="2026-07-07 11:16:04.389561312 +0000 UTC m=+45.928252725"
Jul 07 11:16:04.448539 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.448476    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl" podStartSLOduration=11.739434015 podStartE2EDuration="21.448455078s" totalImagesPullingTime="9.709021063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:04.447766015 +0000 UTC m=+45.986457425" watchObservedRunningTime="2026-07-07 11:16:04.448455078 +0000 UTC m=+45.987146483"
Jul 07 11:16:04.475293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.472358    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cxw5n"]
Jul 07 11:16:04.496248 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.496223    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.502680 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.502291    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 11:16:04.502680 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.502336    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 11:16:04.502680 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.502517    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qdxb4\""
Jul 07 11:16:04.513247 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.513227    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cxw5n"]
Jul 07 11:16:04.562282 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.562252    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:04.572755 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.572706    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d7xvk" podStartSLOduration=11.87534698 podStartE2EDuration="21.572686416s" totalImagesPullingTime="9.697339436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:04.5710018 +0000 UTC m=+46.109693227" watchObservedRunningTime="2026-07-07 11:16:04.572686416 +0000 UTC m=+46.111377830"
Jul 07 11:16:04.573392 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.573362    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.573552 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.573407    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/157703d9-a98d-4586-999e-ef5f7ff78c97-data-volume\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.573601 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.573553    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.573640 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.573602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/157703d9-a98d-4586-999e-ef5f7ff78c97-crio-socket\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.573640 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.573627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwrmf\" (UniqueName: \"kubernetes.io/projected/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-api-access-bwrmf\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.615685 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.615643    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb" podStartSLOduration=11.89146292 podStartE2EDuration="21.615628544s" totalImagesPullingTime="9.724165624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:04.61471861 +0000 UTC m=+46.153410025" watchObservedRunningTime="2026-07-07 11:16:04.615628544 +0000 UTC m=+46.154319956"
Jul 07 11:16:04.674754 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.674730    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.674876 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.674781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/157703d9-a98d-4586-999e-ef5f7ff78c97-crio-socket\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.674876 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:04.674846    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:16:04.674986 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.674890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwrmf\" (UniqueName: \"kubernetes.io/projected/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-api-access-bwrmf\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.674986 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:04.674914    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm podName:157703d9-a98d-4586-999e-ef5f7ff78c97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:05.174893764 +0000 UTC m=+46.713585158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cxw5n" (UID: "157703d9-a98d-4586-999e-ef5f7ff78c97") : configmap "kube-rbac-proxy" not found
Jul 07 11:16:04.674986 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.674959    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/157703d9-a98d-4586-999e-ef5f7ff78c97-crio-socket\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.675139 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.675007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.675139 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.675039    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/157703d9-a98d-4586-999e-ef5f7ff78c97-data-volume\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.675139 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:04.675107    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:16:04.675331 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:04.675152    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls podName:157703d9-a98d-4586-999e-ef5f7ff78c97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:05.17513887 +0000 UTC m=+46.713830260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cxw5n" (UID: "157703d9-a98d-4586-999e-ef5f7ff78c97") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:16:04.675390 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.675371    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/157703d9-a98d-4586-999e-ef5f7ff78c97-data-volume\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.696165 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.696138    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwrmf\" (UniqueName: \"kubernetes.io/projected/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-api-access-bwrmf\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:04.698967 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:04.698944    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:05.068500 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.068467    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:05.171454 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.171413    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:05.179721 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.179688    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:05.179864 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:05.179848    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:16:05.179937 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:05.179904    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:16:05.179937 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:05.179917    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls podName:157703d9-a98d-4586-999e-ef5f7ff78c97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:06.179897992 +0000 UTC m=+47.718589385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cxw5n" (UID: "157703d9-a98d-4586-999e-ef5f7ff78c97") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:16:05.180045 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:05.179952    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm podName:157703d9-a98d-4586-999e-ef5f7ff78c97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:06.179937658 +0000 UTC m=+47.718629053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cxw5n" (UID: "157703d9-a98d-4586-999e-ef5f7ff78c97") : configmap "kube-rbac-proxy" not found
Jul 07 11:16:05.180045 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.179848    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:05.277157 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.277127    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:05.367254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.367175    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:16:05.367660 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.367648    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/0.log"
Jul 07 11:16:05.367709 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.367680    2582 generic.go:356] "Generic (PLEG): container finished" podID="228f0fc4-cb8d-4540-97cb-3f794bbbeb63" containerID="67e1c6ba58d87d36fdb021feb342e58bf202732e22d6e1c1f10b612c31cddb54" exitCode=255
Jul 07 11:16:05.367795 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.367773    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" event={"ID":"228f0fc4-cb8d-4540-97cb-3f794bbbeb63","Type":"ContainerDied","Data":"67e1c6ba58d87d36fdb021feb342e58bf202732e22d6e1c1f10b612c31cddb54"}
Jul 07 11:16:05.367866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.367849    2582 scope.go:117] "RemoveContainer" containerID="0bf72144da08997edb2ba3a6ba0188557eebc9631ac368f17ef797db967ee0ba"
Jul 07 11:16:05.368110 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.368089    2582 scope.go:117] "RemoveContainer" containerID="67e1c6ba58d87d36fdb021feb342e58bf202732e22d6e1c1f10b612c31cddb54"
Jul 07 11:16:05.368337 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:05.368315    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nbjlj_openshift-console-operator(228f0fc4-cb8d-4540-97cb-3f794bbbeb63)\"" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" podUID="228f0fc4-cb8d-4540-97cb-3f794bbbeb63"
Jul 07 11:16:05.371016 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.370994    2582 generic.go:356] "Generic (PLEG): container finished" podID="0266f5e8-632f-4c24-a508-abaa012b9483" containerID="c98d8bf3128ba1cc380c748a07d37a63f069b42c40f18c446f82779c6f89e2c5" exitCode=0
Jul 07 11:16:05.371120 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.371089    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m464b" event={"ID":"0266f5e8-632f-4c24-a508-abaa012b9483","Type":"ContainerDied","Data":"c98d8bf3128ba1cc380c748a07d37a63f069b42c40f18c446f82779c6f89e2c5"}
Jul 07 11:16:05.382225 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.381664    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:05.657614 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.657545    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ksbdk_360ce88e-8a7a-4e3a-bd30-1d2854664c35/dns-node-resolver/0.log"
Jul 07 11:16:05.669706 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.669649    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:05.686081 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.686056    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs"]
Jul 07 11:16:05.704856 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.704825    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs"]
Jul 07 11:16:05.704973 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.704917    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs"
Jul 07 11:16:05.707940 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.707915    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wqn6z\""
Jul 07 11:16:05.708064 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.707916    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 07 11:16:05.708064 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.707994    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 07 11:16:05.764423 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.764394    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:05.786114 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.786094    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxlrm\" (UniqueName: \"kubernetes.io/projected/aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed-kube-api-access-jxlrm\") pod \"migrator-d57d58cc9-6ptxs\" (UID: \"aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs"
Jul 07 11:16:05.871283 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.871259    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:05.887360 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.887329    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxlrm\" (UniqueName: \"kubernetes.io/projected/aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed-kube-api-access-jxlrm\") pod \"migrator-d57d58cc9-6ptxs\" (UID: \"aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs"
Jul 07 11:16:05.905549 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:05.905526    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxlrm\" (UniqueName: \"kubernetes.io/projected/aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed-kube-api-access-jxlrm\") pod \"migrator-d57d58cc9-6ptxs\" (UID: \"aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs"
Jul 07 11:16:06.014887 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.014864    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs"
Jul 07 11:16:06.070463 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.070433    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:06.155702 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.155589    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs"]
Jul 07 11:16:06.157959 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:06.157932    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba4bddb_cae3_4bf7_a7f3_84a8cd2e37ed.slice/crio-dc7bb8f3d41157c14cc790269c5c8e008a805029af0155996bf46e3c026174f5 WatchSource:0}: Error finding container dc7bb8f3d41157c14cc790269c5c8e008a805029af0155996bf46e3c026174f5: Status 404 returned error can't find the container with id dc7bb8f3d41157c14cc790269c5c8e008a805029af0155996bf46e3c026174f5
Jul 07 11:16:06.159614 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.159484    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:06.191442 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.191411    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:06.191525 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.191508    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:06.191561 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:06.191532    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:16:06.191594 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:06.191583    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm podName:157703d9-a98d-4586-999e-ef5f7ff78c97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:08.191564591 +0000 UTC m=+49.730255981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cxw5n" (UID: "157703d9-a98d-4586-999e-ef5f7ff78c97") : configmap "kube-rbac-proxy" not found
Jul 07 11:16:06.191635 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:06.191602    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:16:06.191667 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:06.191644    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls podName:157703d9-a98d-4586-999e-ef5f7ff78c97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:08.191631902 +0000 UTC m=+49.730323295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cxw5n" (UID: "157703d9-a98d-4586-999e-ef5f7ff78c97") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:16:06.234468 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.234420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:06.332950 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.332878    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:06.375465 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.375442    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:16:06.375835 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.375738    2582 scope.go:117] "RemoveContainer" containerID="67e1c6ba58d87d36fdb021feb342e58bf202732e22d6e1c1f10b612c31cddb54"
Jul 07 11:16:06.375929 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:06.375905    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nbjlj_openshift-console-operator(228f0fc4-cb8d-4540-97cb-3f794bbbeb63)\"" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" podUID="228f0fc4-cb8d-4540-97cb-3f794bbbeb63"
Jul 07 11:16:06.376782 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.376760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs" event={"ID":"aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed","Type":"ContainerStarted","Data":"dc7bb8f3d41157c14cc790269c5c8e008a805029af0155996bf46e3c026174f5"}
Jul 07 11:16:06.379837 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.379816    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m464b" event={"ID":"0266f5e8-632f-4c24-a508-abaa012b9483","Type":"ContainerStarted","Data":"d19b993b3e883d6c5f3790fcfae10727d03c214ae786ffb14c96ce9cf69efbfa"}
Jul 07 11:16:06.420986 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.420945    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m464b" podStartSLOduration=9.396073776 podStartE2EDuration="47.420933366s" totalImagesPullingTime="38.02485959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:06.419746992 +0000 UTC m=+47.958438406" watchObservedRunningTime="2026-07-07 11:16:06.420933366 +0000 UTC m=+47.959624760"
Jul 07 11:16:06.454210 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.454173    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-57cf9_1113c816-17c6-479c-bb9e-36ea9b537bdb/node-ca/0.log"
Jul 07 11:16:06.693254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.693158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:06.790274 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.790233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:06.850155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:06.850125    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:07.007239 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.007186    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:07.249421 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.249392    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kmx7n_d99c2e1c-47fd-42ad-ab43-6793dc71a73a/serve-healthcheck-canary/0.log"
Jul 07 11:16:07.400402 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.400314    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:07.501625 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.501590    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:07.573635 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.573601    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-x7nm6"]
Jul 07 11:16:07.592832 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.592784    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-x7nm6"]
Jul 07 11:16:07.592964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.592877    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.595683 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.595658    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 11:16:07.596731 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.596706    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-47fq9\""
Jul 07 11:16:07.596867 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.596740    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 11:16:07.596867 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.596796    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 11:16:07.596867 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.596816    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 11:16:07.614355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.614328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:07.707872 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.707796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/37c0c104-e312-45d3-a2ae-84484f47fac7-signing-key\") pod \"service-ca-758f599c5-x7nm6\" (UID: \"37c0c104-e312-45d3-a2ae-84484f47fac7\") " pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.708012 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.707896    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/37c0c104-e312-45d3-a2ae-84484f47fac7-signing-cabundle\") pod \"service-ca-758f599c5-x7nm6\" (UID: \"37c0c104-e312-45d3-a2ae-84484f47fac7\") " pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.708012 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.707924    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z8sq\" (UniqueName: \"kubernetes.io/projected/37c0c104-e312-45d3-a2ae-84484f47fac7-kube-api-access-5z8sq\") pod \"service-ca-758f599c5-x7nm6\" (UID: \"37c0c104-e312-45d3-a2ae-84484f47fac7\") " pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.711340 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.711315    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:07.809175 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.809145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/37c0c104-e312-45d3-a2ae-84484f47fac7-signing-key\") pod \"service-ca-758f599c5-x7nm6\" (UID: \"37c0c104-e312-45d3-a2ae-84484f47fac7\") " pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.809334 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.809294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/37c0c104-e312-45d3-a2ae-84484f47fac7-signing-cabundle\") pod \"service-ca-758f599c5-x7nm6\" (UID: \"37c0c104-e312-45d3-a2ae-84484f47fac7\") " pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.809403 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.809336    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z8sq\" (UniqueName: \"kubernetes.io/projected/37c0c104-e312-45d3-a2ae-84484f47fac7-kube-api-access-5z8sq\") pod \"service-ca-758f599c5-x7nm6\" (UID: \"37c0c104-e312-45d3-a2ae-84484f47fac7\") " pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.809900 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.809875    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/37c0c104-e312-45d3-a2ae-84484f47fac7-signing-cabundle\") pod \"service-ca-758f599c5-x7nm6\" (UID: \"37c0c104-e312-45d3-a2ae-84484f47fac7\") " pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.811465 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.811442    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/37c0c104-e312-45d3-a2ae-84484f47fac7-signing-key\") pod \"service-ca-758f599c5-x7nm6\" (UID: \"37c0c104-e312-45d3-a2ae-84484f47fac7\") " pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.817844 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.817826    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z8sq\" (UniqueName: \"kubernetes.io/projected/37c0c104-e312-45d3-a2ae-84484f47fac7-kube-api-access-5z8sq\") pod \"service-ca-758f599c5-x7nm6\" (UID: \"37c0c104-e312-45d3-a2ae-84484f47fac7\") " pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.903571 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.903544    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-x7nm6"
Jul 07 11:16:07.949511 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:07.949482    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:08.212691 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.212665    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:08.212801 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.212784    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:08.212874 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.212798    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:16:08.212874 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.212872    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm podName:157703d9-a98d-4586-999e-ef5f7ff78c97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:12.212850625 +0000 UTC m=+53.751542035 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cxw5n" (UID: "157703d9-a98d-4586-999e-ef5f7ff78c97") : configmap "kube-rbac-proxy" not found
Jul 07 11:16:08.212982 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.212970    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:16:08.213057 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.213034    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls podName:157703d9-a98d-4586-999e-ef5f7ff78c97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:12.213014676 +0000 UTC m=+53.751706090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cxw5n" (UID: "157703d9-a98d-4586-999e-ef5f7ff78c97") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:16:08.262029 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.262008    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-x7nm6"]
Jul 07 11:16:08.264348 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:08.264324    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37c0c104_e312_45d3_a2ae_84484f47fac7.slice/crio-3209269e01b2fac39086758fe0391c291c9cf34e11f769e90fb49a42d9a8b529 WatchSource:0}: Error finding container 3209269e01b2fac39086758fe0391c291c9cf34e11f769e90fb49a42d9a8b529: Status 404 returned error can't find the container with id 3209269e01b2fac39086758fe0391c291c9cf34e11f769e90fb49a42d9a8b529
Jul 07 11:16:08.387060 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.387029    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs" event={"ID":"aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed","Type":"ContainerStarted","Data":"695a0b8469c06b870ea1e07805b0ca1e78c226fde4b33c46c51a8c9c45248d3b"}
Jul 07 11:16:08.388745 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.388722    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-x7nm6" event={"ID":"37c0c104-e312-45d3-a2ae-84484f47fac7","Type":"ContainerStarted","Data":"16bb5e2460eea16c438a4cacb3483dd550f2e02ddaa8d6da3b898ddefd152e31"}
Jul 07 11:16:08.388823 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.388762    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-x7nm6" event={"ID":"37c0c104-e312-45d3-a2ae-84484f47fac7","Type":"ContainerStarted","Data":"3209269e01b2fac39086758fe0391c291c9cf34e11f769e90fb49a42d9a8b529"}
Jul 07 11:16:08.504055 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.504024    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:08.716927 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.716896    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:16:08.717089 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.716947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:16:08.717089 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.716988    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:16:08.717089 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.717042    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:16:08.717288 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.717101    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls podName:cc5862d5-881b-445b-9abf-93f146315de8 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:24.7170861 +0000 UTC m=+66.255777489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls") pod "dns-default-wxjtb" (UID: "cc5862d5-881b-445b-9abf-93f146315de8") : secret "dns-default-metrics-tls" not found
Jul 07 11:16:08.717288 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.717146    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:16:08.717288 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.717163    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85d77785b6-h2klp: secret "image-registry-tls" not found
Jul 07 11:16:08.717288 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.717169    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:16:08.717288 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.717242    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls podName:a58bd4ae-21c5-427d-a2e3-0791f820c597 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:24.717224481 +0000 UTC m=+66.255915888 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls") pod "image-registry-85d77785b6-h2klp" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597") : secret "image-registry-tls" not found
Jul 07 11:16:08.717288 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.717263    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls podName:ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:24.717252654 +0000 UTC m=+66.255944044 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-rf5tn" (UID: "ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56") : secret "samples-operator-tls" not found
Jul 07 11:16:08.818021 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.817992    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:16:08.818166 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.818043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:08.818166 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.818136    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:16:08.818166 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.818159    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:08.818331 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.818178    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:16:08.818331 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.818143    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:16:08.818331 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.818212    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert podName:9b40451d-1272-4a5d-a8be-fa8a20b6b565 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:24.818177476 +0000 UTC m=+66.356868866 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wshqb" (UID: "9b40451d-1272-4a5d-a8be-fa8a20b6b565") : secret "networking-console-plugin-cert" not found
Jul 07 11:16:08.818331 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.818243    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:24.818231503 +0000 UTC m=+66.356922914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : secret "router-metrics-certs-default" not found
Jul 07 11:16:08.818331 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.818248    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:16:08.818331 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.818295    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls podName:1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:24.818282327 +0000 UTC m=+66.356973729 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-k46bt" (UID: "1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:16:08.818603 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:08.818358    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle podName:1de016bd-1463-4d64-ad1b-09e17dce5d18 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:24.818346031 +0000 UTC m=+66.357037425 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle") pod "router-default-5c5dbcc7d5-crvzt" (UID: "1de016bd-1463-4d64-ad1b-09e17dce5d18") : configmap references non-existent config key: service-ca.crt
Jul 07 11:16:08.876058 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:08.876031    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:09.393885 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:09.393844    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs" event={"ID":"aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed","Type":"ContainerStarted","Data":"072e1c32aabed49b221f7dbd4c0404e541dc09be82d1d269f72f1ac887e92dc5"}
Jul 07 11:16:09.423484 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:09.423422    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6ptxs" podStartSLOduration=2.453685154 podStartE2EDuration="4.423404205s" totalImagesPullingTime="1.969719051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:09.418331516 +0000 UTC m=+50.957022928" watchObservedRunningTime="2026-07-07 11:16:09.423404205 +0000 UTC m=+50.962095657"
Jul 07 11:16:09.453834 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:09.453791    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-x7nm6" podStartSLOduration=2.453778017 podStartE2EDuration="2.453778017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:09.453556844 +0000 UTC m=+50.992248258" watchObservedRunningTime="2026-07-07 11:16:09.453778017 +0000 UTC m=+50.992469429"
Jul 07 11:16:09.834096 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:09.834057    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:09.954127 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:09.954091    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:10.085941 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:10.085791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:10.370893 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:10.370801    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:10.457588 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:10.457545    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:10.546519 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:10.546482    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:11.711413 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:11.711376    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:11.907283 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:11.907247    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:12.144833 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:12.144803    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:12.248302 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:12.248277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:12.248462 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:12.248352    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:12.248462 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:12.248452    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:16:12.248534 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:12.248504    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls podName:157703d9-a98d-4586-999e-ef5f7ff78c97 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:20.248491027 +0000 UTC m=+61.787182417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cxw5n" (UID: "157703d9-a98d-4586-999e-ef5f7ff78c97") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:16:12.248827 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:12.248802    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/157703d9-a98d-4586-999e-ef5f7ff78c97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:12.320656 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:12.320626    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:12.504259 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:12.504229    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:12.566293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:12.566264    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:13.200451 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:13.200422    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:16:13.200984 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:13.200885    2582 scope.go:117] "RemoveContainer" containerID="67e1c6ba58d87d36fdb021feb342e58bf202732e22d6e1c1f10b612c31cddb54"
Jul 07 11:16:13.201094 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:13.201072    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nbjlj_openshift-console-operator(228f0fc4-cb8d-4540-97cb-3f794bbbeb63)\"" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" podUID="228f0fc4-cb8d-4540-97cb-3f794bbbeb63"
Jul 07 11:16:13.380867 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:13.380828    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:16:13.406612 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:13.406589    2582 scope.go:117] "RemoveContainer" containerID="67e1c6ba58d87d36fdb021feb342e58bf202732e22d6e1c1f10b612c31cddb54"
Jul 07 11:16:13.406755 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:13.406737    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nbjlj_openshift-console-operator(228f0fc4-cb8d-4540-97cb-3f794bbbeb63)\"" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" podUID="228f0fc4-cb8d-4540-97cb-3f794bbbeb63"
Jul 07 11:16:13.530230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:13.530188    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:13.628906 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:13.628873    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:13.734834 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:13.734802    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:13.918448 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:13.918356    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:14.107270 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:14.107230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:14.316741 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:14.316706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:14.502002 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:14.501969    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:15.696703 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:15.696669    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:15.893800 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:15.893761    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:16.017584 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:16.017555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:16.162606 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:16.162568    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:16.301144 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:16.301054    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:16.509448 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:16.509418    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:17.502895 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:17.502864    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:17.709928 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:17.709890    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:18.098794 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:18.098762    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:18.270546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:18.270522    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6fb5x"
Jul 07 11:16:18.305537 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:18.305503    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:18.907590 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:18.907551    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:19.304819 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:19.304785    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:19.708003 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:19.707922    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:19.903984 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:19.903946    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:19.978782 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:19.978754    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:20.075815 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:20.075783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:20.177666 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:20.177632    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:20.321918 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:20.321840    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:20.324001 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:20.323976    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/157703d9-a98d-4586-999e-ef5f7ff78c97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cxw5n\" (UID: \"157703d9-a98d-4586-999e-ef5f7ff78c97\") " pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:20.357934 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:20.357898    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:20.416715 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:20.416692    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qdxb4\""
Jul 07 11:16:20.424378 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:20.424357    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cxw5n"
Jul 07 11:16:20.524032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:20.524006    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:20.558459 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:20.558432    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cxw5n"]
Jul 07 11:16:20.561364 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:20.561333    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod157703d9_a98d_4586_999e_ef5f7ff78c97.slice/crio-ebd4701d93840a3bab2560f8c9067e832c46d9d3d3406f7dc9cb9326e8b14e53 WatchSource:0}: Error finding container ebd4701d93840a3bab2560f8c9067e832c46d9d3d3406f7dc9cb9326e8b14e53: Status 404 returned error can't find the container with id ebd4701d93840a3bab2560f8c9067e832c46d9d3d3406f7dc9cb9326e8b14e53
Jul 07 11:16:21.434957 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:21.434880    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cxw5n" event={"ID":"157703d9-a98d-4586-999e-ef5f7ff78c97","Type":"ContainerStarted","Data":"ebd4701d93840a3bab2560f8c9067e832c46d9d3d3406f7dc9cb9326e8b14e53"}
Jul 07 11:16:23.654724 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.654687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:16:23.657663 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.657641    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 11:16:23.668175 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.668147    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca-metrics-certs\") pod \"network-metrics-daemon-rfz9v\" (UID: \"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca\") " pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:16:23.734099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.734071    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89jkb\""
Jul 07 11:16:23.741536 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.741515    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rfz9v"
Jul 07 11:16:23.756090 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.756069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:16:23.759347 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.759327    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 11:16:23.769014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.768987    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/96412d1b-55ef-4558-88c3-4b83911151b5-original-pull-secret\") pod \"global-pull-secret-syncer-cb8jb\" (UID: \"96412d1b-55ef-4558-88c3-4b83911151b5\") " pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:16:23.857311 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.857276    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldcr\" (UniqueName: \"kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr\") pod \"network-check-target-524zs\" (UID: \"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2\") " pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:16:23.859902 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.859877    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ldcr\" (UniqueName: \"kubernetes.io/projected/4c33a0a8-99d2-4472-8aac-3b9cd92eaee2-kube-api-access-4ldcr\") pod \"network-check-target-524zs\" (UID: \"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2\") " pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:16:23.932943 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.932875    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42fbv\""
Jul 07 11:16:23.939712 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:23.939691    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:16:24.037734 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.037707    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cb8jb"
Jul 07 11:16:24.549740 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.549716    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cb8jb"]
Jul 07 11:16:24.550654 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:24.550628    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96412d1b_55ef_4558_88c3_4b83911151b5.slice/crio-cd0559d347e40c614082066c726bfbd06fb78cf56ce92c14e7e8ab9c781103b9 WatchSource:0}: Error finding container cd0559d347e40c614082066c726bfbd06fb78cf56ce92c14e7e8ab9c781103b9: Status 404 returned error can't find the container with id cd0559d347e40c614082066c726bfbd06fb78cf56ce92c14e7e8ab9c781103b9
Jul 07 11:16:24.768362 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.768333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:16:24.768856 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.768381    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:16:24.768856 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.768446    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:16:24.770534 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.770511    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc5862d5-881b-445b-9abf-93f146315de8-metrics-tls\") pod \"dns-default-wxjtb\" (UID: \"cc5862d5-881b-445b-9abf-93f146315de8\") " pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:16:24.770645 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.770625    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") pod \"image-registry-85d77785b6-h2klp\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") " pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:16:24.770704 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.770647    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-rf5tn\" (UID: \"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:16:24.774570 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.774551    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-524zs"]
Jul 07 11:16:24.776899 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:24.776872    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c33a0a8_99d2_4472_8aac_3b9cd92eaee2.slice/crio-ee6c8fc86745f73a7a4cf148fccba22105be3f25284105a4cf532cd78180b2b2 WatchSource:0}: Error finding container ee6c8fc86745f73a7a4cf148fccba22105be3f25284105a4cf532cd78180b2b2: Status 404 returned error can't find the container with id ee6c8fc86745f73a7a4cf148fccba22105be3f25284105a4cf532cd78180b2b2
Jul 07 11:16:24.782769 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.782707    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rfz9v"]
Jul 07 11:16:24.786327 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:24.786305    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a63e931_4e3a_4e6d_9a17_b0a6dd83ccca.slice/crio-860dc8c450b89a5bb799493823bba2508c10859e8715e945b9d6c927922bf23e WatchSource:0}: Error finding container 860dc8c450b89a5bb799493823bba2508c10859e8715e945b9d6c927922bf23e: Status 404 returned error can't find the container with id 860dc8c450b89a5bb799493823bba2508c10859e8715e945b9d6c927922bf23e
Jul 07 11:16:24.868846 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.868818    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:16:24.868941 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.868858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:24.868941 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.868893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:24.868941 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.868911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:16:24.869598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.869579    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1de016bd-1463-4d64-ad1b-09e17dce5d18-service-ca-bundle\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:24.870944 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.870922    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1de016bd-1463-4d64-ad1b-09e17dce5d18-metrics-certs\") pod \"router-default-5c5dbcc7d5-crvzt\" (UID: \"1de016bd-1463-4d64-ad1b-09e17dce5d18\") " pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:24.871045 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.871024    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b40451d-1272-4a5d-a8be-fa8a20b6b565-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wshqb\" (UID: \"9b40451d-1272-4a5d-a8be-fa8a20b6b565\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:16:24.871100 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.871051    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-k46bt\" (UID: \"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:16:24.893576 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.893552    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:24.928389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.928372    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rbld5\""
Jul 07 11:16:24.936616 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.936602    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:16:24.941415 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.941394    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9mmvf\""
Jul 07 11:16:24.949182 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.949164    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"
Jul 07 11:16:24.954130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.954102    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgd79\""
Jul 07 11:16:24.961651 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.961630    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:16:24.980830 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:24.980802    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:25.020604 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.020365    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-t4bj8\""
Jul 07 11:16:25.027393 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.027369    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:25.031110 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.030903    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rnrbp\""
Jul 07 11:16:25.038593 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.038573    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"
Jul 07 11:16:25.067735 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.067670    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4mdc9\""
Jul 07 11:16:25.074995 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.074757    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"
Jul 07 11:16:25.087515 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.087485    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85d77785b6-h2klp"]
Jul 07 11:16:25.098335 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:25.097590    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda58bd4ae_21c5_427d_a2e3_0791f820c597.slice/crio-a4e57e44060267be5e447a55492d920ab5d91e0aeb39d8f277aaa9d26ca30a9a WatchSource:0}: Error finding container a4e57e44060267be5e447a55492d920ab5d91e0aeb39d8f277aaa9d26ca30a9a: Status 404 returned error can't find the container with id a4e57e44060267be5e447a55492d920ab5d91e0aeb39d8f277aaa9d26ca30a9a
Jul 07 11:16:25.117133 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.116813    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn"]
Jul 07 11:16:25.147858 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.147493    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wxjtb"]
Jul 07 11:16:25.148129 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:25.148083    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5862d5_881b_445b_9abf_93f146315de8.slice/crio-f35d2c5580f67709ee3a5a1f5d1e184f87f44ee5e2769491e645f1df183f9ebc WatchSource:0}: Error finding container f35d2c5580f67709ee3a5a1f5d1e184f87f44ee5e2769491e645f1df183f9ebc: Status 404 returned error can't find the container with id f35d2c5580f67709ee3a5a1f5d1e184f87f44ee5e2769491e645f1df183f9ebc
Jul 07 11:16:25.215327 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.215057    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c5dbcc7d5-crvzt"]
Jul 07 11:16:25.215564 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:25.215525    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de016bd_1463_4d64_ad1b_09e17dce5d18.slice/crio-de821f3864cff9519172879e998da8a6467dd7ccc40744164396b08b3b1e7ead WatchSource:0}: Error finding container de821f3864cff9519172879e998da8a6467dd7ccc40744164396b08b3b1e7ead: Status 404 returned error can't find the container with id de821f3864cff9519172879e998da8a6467dd7ccc40744164396b08b3b1e7ead
Jul 07 11:16:25.234868 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.233000    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt"]
Jul 07 11:16:25.237834 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:25.237796    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dbeb02c_2837_40aa_bdc1_e6f3a03a4c46.slice/crio-166b7afcaa7152f1cfb951d88a87cd5985613c2b8b8c280a758a78659c793177 WatchSource:0}: Error finding container 166b7afcaa7152f1cfb951d88a87cd5985613c2b8b8c280a758a78659c793177: Status 404 returned error can't find the container with id 166b7afcaa7152f1cfb951d88a87cd5985613c2b8b8c280a758a78659c793177
Jul 07 11:16:25.263672 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.263613    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wshqb"]
Jul 07 11:16:25.269761 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:25.269664    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b40451d_1272_4a5d_a8be_fa8a20b6b565.slice/crio-06b795f299926a7d2f400dbcc5bea8d9b7a2e0d3c81985b35518633be73ab444 WatchSource:0}: Error finding container 06b795f299926a7d2f400dbcc5bea8d9b7a2e0d3c81985b35518633be73ab444: Status 404 returned error can't find the container with id 06b795f299926a7d2f400dbcc5bea8d9b7a2e0d3c81985b35518633be73ab444
Jul 07 11:16:25.454681 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.454639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn" event={"ID":"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56","Type":"ContainerStarted","Data":"e9b2847a24462558696affcb247486a71d8859d340903dfabdbc196ebe1cfc1c"}
Jul 07 11:16:25.456531 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.456502    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85d77785b6-h2klp" event={"ID":"a58bd4ae-21c5-427d-a2e3-0791f820c597","Type":"ContainerStarted","Data":"050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793"}
Jul 07 11:16:25.456673 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.456552    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85d77785b6-h2klp" event={"ID":"a58bd4ae-21c5-427d-a2e3-0791f820c597","Type":"ContainerStarted","Data":"a4e57e44060267be5e447a55492d920ab5d91e0aeb39d8f277aaa9d26ca30a9a"}
Jul 07 11:16:25.456673 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.456604    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:16:25.458499 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.458429    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-524zs" event={"ID":"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2","Type":"ContainerStarted","Data":"27a6ac4154b768bb70a957e0691831ab374bb270fb22031ce69e995b01a21c9c"}
Jul 07 11:16:25.458499 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.458479    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:16:25.458499 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.458500    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-524zs" event={"ID":"4c33a0a8-99d2-4472-8aac-3b9cd92eaee2","Type":"ContainerStarted","Data":"ee6c8fc86745f73a7a4cf148fccba22105be3f25284105a4cf532cd78180b2b2"}
Jul 07 11:16:25.460658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.460616    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb" event={"ID":"9b40451d-1272-4a5d-a8be-fa8a20b6b565","Type":"ContainerStarted","Data":"06b795f299926a7d2f400dbcc5bea8d9b7a2e0d3c81985b35518633be73ab444"}
Jul 07 11:16:25.462189 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.462139    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wxjtb" event={"ID":"cc5862d5-881b-445b-9abf-93f146315de8","Type":"ContainerStarted","Data":"f35d2c5580f67709ee3a5a1f5d1e184f87f44ee5e2769491e645f1df183f9ebc"}
Jul 07 11:16:25.463977 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.463945    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cb8jb" event={"ID":"96412d1b-55ef-4558-88c3-4b83911151b5","Type":"ContainerStarted","Data":"cd0559d347e40c614082066c726bfbd06fb78cf56ce92c14e7e8ab9c781103b9"}
Jul 07 11:16:25.478267 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.478232    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rfz9v" event={"ID":"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca","Type":"ContainerStarted","Data":"860dc8c450b89a5bb799493823bba2508c10859e8715e945b9d6c927922bf23e"}
Jul 07 11:16:25.479929 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.479903    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt" event={"ID":"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46","Type":"ContainerStarted","Data":"166b7afcaa7152f1cfb951d88a87cd5985613c2b8b8c280a758a78659c793177"}
Jul 07 11:16:25.481771 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.481734    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt" event={"ID":"1de016bd-1463-4d64-ad1b-09e17dce5d18","Type":"ContainerStarted","Data":"b1531d640d41fdfa6d6b6a6bd45f838616856928b8221fa23c230d7ffa015b04"}
Jul 07 11:16:25.481842 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.481774    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt" event={"ID":"1de016bd-1463-4d64-ad1b-09e17dce5d18","Type":"ContainerStarted","Data":"de821f3864cff9519172879e998da8a6467dd7ccc40744164396b08b3b1e7ead"}
Jul 07 11:16:25.486701 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.486666    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cxw5n" event={"ID":"157703d9-a98d-4586-999e-ef5f7ff78c97","Type":"ContainerStarted","Data":"072a7cdf1123e47a3cecd949edf074791c16ccb0617fcfc17829de61cdd88fe0"}
Jul 07 11:16:25.524397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.524221    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85d77785b6-h2klp" podStartSLOduration=66.52417273 podStartE2EDuration="1m6.52417273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:25.515371732 +0000 UTC m=+67.054063181" watchObservedRunningTime="2026-07-07 11:16:25.52417273 +0000 UTC m=+67.062864142"
Jul 07 11:16:25.549791 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.549729    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-524zs" podStartSLOduration=66.54970768 podStartE2EDuration="1m6.54970768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:25.548465638 +0000 UTC m=+67.087157051" watchObservedRunningTime="2026-07-07 11:16:25.54970768 +0000 UTC m=+67.088399093"
Jul 07 11:16:25.583178 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:25.583104    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt" podStartSLOduration=42.583085369 podStartE2EDuration="42.583085369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:25.581837536 +0000 UTC m=+67.120528949" watchObservedRunningTime="2026-07-07 11:16:25.583085369 +0000 UTC m=+67.121776783"
Jul 07 11:16:26.028578 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:26.028543    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:26.031689 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:26.031662    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:26.493576 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:26.493528    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rfz9v" event={"ID":"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca","Type":"ContainerStarted","Data":"1890ff60235fa6715fb275b5defc3f2dd0998c22dccffaa6b837a687148a2a47"}
Jul 07 11:16:26.498444 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:26.498351    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cxw5n" event={"ID":"157703d9-a98d-4586-999e-ef5f7ff78c97","Type":"ContainerStarted","Data":"d40d430c6106928547ec697f239b51e104027beb7b9617b61590eabf0fc9972b"}
Jul 07 11:16:26.498583 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:26.498543    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:26.499799 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:26.499778    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5c5dbcc7d5-crvzt"
Jul 07 11:16:27.505826 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:27.505743    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rfz9v" event={"ID":"0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca","Type":"ContainerStarted","Data":"9a2a498b2efd50aa202f92bd88e99ad6a7ae6d2b2501f5af000fe38f42718226"}
Jul 07 11:16:27.524996 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:27.524936    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rfz9v" podStartSLOduration=67.054930494 podStartE2EDuration="1m8.524914534s" totalImagesPullingTime="1.46998404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:27.523044514 +0000 UTC m=+69.061735927" watchObservedRunningTime="2026-07-07 11:16:27.524914534 +0000 UTC m=+69.063605948"
Jul 07 11:16:28.019441 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.019410    2582 scope.go:117] "RemoveContainer" containerID="67e1c6ba58d87d36fdb021feb342e58bf202732e22d6e1c1f10b612c31cddb54"
Jul 07 11:16:28.657252 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.657150    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"]
Jul 07 11:16:28.691372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.691330    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"]
Jul 07 11:16:28.693891 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.693655    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:28.698443 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.698392    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 11:16:28.700462 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.700436    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 11:16:28.700735 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.700721    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 11:16:28.701291 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.701273    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 07 11:16:28.711176 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.711154    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"]
Jul 07 11:16:28.711353 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.711324    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.714107 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.714046    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 07 11:16:28.714228 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.714183    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 07 11:16:28.714467 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.714449    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 07 11:16:28.714710 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.714692    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 07 11:16:28.731301 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.731276    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"]
Jul 07 11:16:28.731301 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.731297    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"
Jul 07 11:16:28.731464 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.731375    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"]
Jul 07 11:16:28.731464 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.731396    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"]
Jul 07 11:16:28.733771 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.733751    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-s858m\""
Jul 07 11:16:28.734227 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.734036    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 07 11:16:28.741538 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.741514    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:28.803830 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.803801    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-ca\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.803986 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.803851    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6569d98-137a-4f0c-a398-790d08fc1b96-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.803986 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.803886    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-hub\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.803986 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.803916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6vqf\" (UniqueName: \"kubernetes.io/projected/02d71c7f-00a6-489f-b769-906441000bec-kube-api-access-j6vqf\") pod \"managed-serviceaccount-addon-agent-7bcb645d76-kph6j\" (UID: \"02d71c7f-00a6-489f-b769-906441000bec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"
Jul 07 11:16:28.803986 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.803934    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttgcn\" (UniqueName: \"kubernetes.io/projected/c6569d98-137a-4f0c-a398-790d08fc1b96-kube-api-access-ttgcn\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.804180 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.804042    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjch7\" (UniqueName: \"kubernetes.io/projected/a3f38aee-83a7-41ac-ba0b-680edf736cd4-kube-api-access-pjch7\") pod \"klusterlet-addon-workmgr-7cbffb6b9c-gcfp9\" (UID: \"a3f38aee-83a7-41ac-ba0b-680edf736cd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:28.804180 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.804085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02d71c7f-00a6-489f-b769-906441000bec-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bcb645d76-kph6j\" (UID: \"02d71c7f-00a6-489f-b769-906441000bec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"
Jul 07 11:16:28.804180 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.804116    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3f38aee-83a7-41ac-ba0b-680edf736cd4-tmp\") pod \"klusterlet-addon-workmgr-7cbffb6b9c-gcfp9\" (UID: \"a3f38aee-83a7-41ac-ba0b-680edf736cd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:28.804180 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.804142    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.804180 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.804165    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3f38aee-83a7-41ac-ba0b-680edf736cd4-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cbffb6b9c-gcfp9\" (UID: \"a3f38aee-83a7-41ac-ba0b-680edf736cd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:28.804435 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.804266    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.905632 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.905599    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3f38aee-83a7-41ac-ba0b-680edf736cd4-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cbffb6b9c-gcfp9\" (UID: \"a3f38aee-83a7-41ac-ba0b-680edf736cd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:28.905803 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.905746    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.905878 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.905822    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-ca\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.905878 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.905858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6569d98-137a-4f0c-a398-790d08fc1b96-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.905989 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.905883    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-hub\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.905989 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.905915    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6vqf\" (UniqueName: \"kubernetes.io/projected/02d71c7f-00a6-489f-b769-906441000bec-kube-api-access-j6vqf\") pod \"managed-serviceaccount-addon-agent-7bcb645d76-kph6j\" (UID: \"02d71c7f-00a6-489f-b769-906441000bec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"
Jul 07 11:16:28.905989 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.905940    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttgcn\" (UniqueName: \"kubernetes.io/projected/c6569d98-137a-4f0c-a398-790d08fc1b96-kube-api-access-ttgcn\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.906133 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.906005    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjch7\" (UniqueName: \"kubernetes.io/projected/a3f38aee-83a7-41ac-ba0b-680edf736cd4-kube-api-access-pjch7\") pod \"klusterlet-addon-workmgr-7cbffb6b9c-gcfp9\" (UID: \"a3f38aee-83a7-41ac-ba0b-680edf736cd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:28.906133 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.906027    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02d71c7f-00a6-489f-b769-906441000bec-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bcb645d76-kph6j\" (UID: \"02d71c7f-00a6-489f-b769-906441000bec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"
Jul 07 11:16:28.906133 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.906052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3f38aee-83a7-41ac-ba0b-680edf736cd4-tmp\") pod \"klusterlet-addon-workmgr-7cbffb6b9c-gcfp9\" (UID: \"a3f38aee-83a7-41ac-ba0b-680edf736cd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:28.906133 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.906092    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.906996 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.906931    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6569d98-137a-4f0c-a398-790d08fc1b96-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.907117 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.907070    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3f38aee-83a7-41ac-ba0b-680edf736cd4-tmp\") pod \"klusterlet-addon-workmgr-7cbffb6b9c-gcfp9\" (UID: \"a3f38aee-83a7-41ac-ba0b-680edf736cd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:28.909400 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.909321    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02d71c7f-00a6-489f-b769-906441000bec-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bcb645d76-kph6j\" (UID: \"02d71c7f-00a6-489f-b769-906441000bec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"
Jul 07 11:16:28.909508 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.909400    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-ca\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.909877 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.909833    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.910096 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.910070    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.910187 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.910099    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c6569d98-137a-4f0c-a398-790d08fc1b96-hub\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:28.910187 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.910118    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3f38aee-83a7-41ac-ba0b-680edf736cd4-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cbffb6b9c-gcfp9\" (UID: \"a3f38aee-83a7-41ac-ba0b-680edf736cd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:28.917701 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.917676    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:28.926417 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.926393    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6vqf\" (UniqueName: \"kubernetes.io/projected/02d71c7f-00a6-489f-b769-906441000bec-kube-api-access-j6vqf\") pod \"managed-serviceaccount-addon-agent-7bcb645d76-kph6j\" (UID: \"02d71c7f-00a6-489f-b769-906441000bec\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"
Jul 07 11:16:28.928451 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.928431    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjch7\" (UniqueName: \"kubernetes.io/projected/a3f38aee-83a7-41ac-ba0b-680edf736cd4-kube-api-access-pjch7\") pod \"klusterlet-addon-workmgr-7cbffb6b9c-gcfp9\" (UID: \"a3f38aee-83a7-41ac-ba0b-680edf736cd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:28.934960 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:28.934939    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttgcn\" (UniqueName: \"kubernetes.io/projected/c6569d98-137a-4f0c-a398-790d08fc1b96-kube-api-access-ttgcn\") pod \"cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b\" (UID: \"c6569d98-137a-4f0c-a398-790d08fc1b96\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:29.006219 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:29.006150    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:29.033131 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:29.033106    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"
Jul 07 11:16:29.041407 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:29.041384    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"
Jul 07 11:16:29.125241 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:29.125216    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:29.312318 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:29.312288    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:32.833558 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:32.833022    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b"]
Jul 07 11:16:32.861306 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:32.861238    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j"]
Jul 07 11:16:32.871419 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:32.871391    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02d71c7f_00a6_489f_b769_906441000bec.slice/crio-5e3e1a26f27da8fbf808af2be401c4f6ab130cda213591a52c92084dc791aad8 WatchSource:0}: Error finding container 5e3e1a26f27da8fbf808af2be401c4f6ab130cda213591a52c92084dc791aad8: Status 404 returned error can't find the container with id 5e3e1a26f27da8fbf808af2be401c4f6ab130cda213591a52c92084dc791aad8
Jul 07 11:16:32.876312 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:32.876155    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"]
Jul 07 11:16:32.888424 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:32.888387    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f38aee_83a7_41ac_ba0b_680edf736cd4.slice/crio-2ea79646e1eecbe302adc35c085662830f1d9aa266c345f5fa9be51b269106db WatchSource:0}: Error finding container 2ea79646e1eecbe302adc35c085662830f1d9aa266c345f5fa9be51b269106db: Status 404 returned error can't find the container with id 2ea79646e1eecbe302adc35c085662830f1d9aa266c345f5fa9be51b269106db
Jul 07 11:16:33.177167 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.177133    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:33.385459 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.385369    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:33.480559 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.480525    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:33.529487 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.529437    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt" event={"ID":"1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46","Type":"ContainerStarted","Data":"d989ae025ce17c1a831dbf6bae6d7f4bc2d4bfb94332bc0aed2daf6a18ab871e"}
Jul 07 11:16:33.531630 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.531603    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cxw5n" event={"ID":"157703d9-a98d-4586-999e-ef5f7ff78c97","Type":"ContainerStarted","Data":"c59b8f14561d5a3bedf2ba9e007c5e4138f6e178faaefcc33742c784073c8697"}
Jul 07 11:16:33.533668 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.533648    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:16:33.533794 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.533731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj" event={"ID":"228f0fc4-cb8d-4540-97cb-3f794bbbeb63","Type":"ContainerStarted","Data":"e5d5964e27e184d8689a73f10c637e76a5187fc4c560654ff27befffb19419ec"}
Jul 07 11:16:33.534031 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.534012    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:16:33.534979 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.534956    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j" event={"ID":"02d71c7f-00a6-489f-b769-906441000bec","Type":"ContainerStarted","Data":"5e3e1a26f27da8fbf808af2be401c4f6ab130cda213591a52c92084dc791aad8"}
Jul 07 11:16:33.536892 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.536865    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn" event={"ID":"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56","Type":"ContainerStarted","Data":"e3c44155a25e10e85edadf9ab19102254c90c71e0e8950c3ce8bfd98e1afcf47"}
Jul 07 11:16:33.536993 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.536908    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn" event={"ID":"ebf73fe1-a4f9-4bb2-8a03-4a240c06ea56","Type":"ContainerStarted","Data":"02413cbde7be656af095906cd8113b1dff1888f428046a4ace74e3c4d6d0e703"}
Jul 07 11:16:33.538000 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.537966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9" event={"ID":"a3f38aee-83a7-41ac-ba0b-680edf736cd4","Type":"ContainerStarted","Data":"2ea79646e1eecbe302adc35c085662830f1d9aa266c345f5fa9be51b269106db"}
Jul 07 11:16:33.539786 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.539756    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb" event={"ID":"9b40451d-1272-4a5d-a8be-fa8a20b6b565","Type":"ContainerStarted","Data":"1af4f1cf6db782bd1888d30d96ee6d4b7fee7fbac6d9b3742cc5504551725b90"}
Jul 07 11:16:33.541250 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.541225    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-nbjlj"
Jul 07 11:16:33.546346 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.546320    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wxjtb" event={"ID":"cc5862d5-881b-445b-9abf-93f146315de8","Type":"ContainerStarted","Data":"c45270c7fa056a0d227db39e15db12f69f4836ea1f8e0f18b07af6f8d68584dd"}
Jul 07 11:16:33.546434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.546364    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wxjtb" event={"ID":"cc5862d5-881b-445b-9abf-93f146315de8","Type":"ContainerStarted","Data":"241caee76ae5cdaf91cc15f2dde4c2c83f71f54585c1342c0fb39fd2d18d253d"}
Jul 07 11:16:33.546434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.546424    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:16:33.548078 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.548031    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-k46bt" podStartSLOduration=42.183245687 podStartE2EDuration="49.548015648s" totalImagesPullingTime="7.364769961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:33.546844695 +0000 UTC m=+75.085536108" watchObservedRunningTime="2026-07-07 11:16:33.548015648 +0000 UTC m=+75.086707062"
Jul 07 11:16:33.548462 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.548422    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cb8jb" event={"ID":"96412d1b-55ef-4558-88c3-4b83911151b5","Type":"ContainerStarted","Data":"541d5d7336464ff911a6238bcc315779e3dd431005507a5e68c23ec51d0a4f1e"}
Jul 07 11:16:33.549755 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.549715    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b" event={"ID":"c6569d98-137a-4f0c-a398-790d08fc1b96","Type":"ContainerStarted","Data":"9bb6501674b6de68c36fa7982706e9032cdbbae92ab455e866c622f7250774fb"}
Jul 07 11:16:33.552238 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.552171    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:33.595737 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.595684    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wshqb" podStartSLOduration=39.793741236 podStartE2EDuration="41.595666856s" totalImagesPullingTime="1.80192562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:33.595120871 +0000 UTC m=+75.133812284" watchObservedRunningTime="2026-07-07 11:16:33.595666856 +0000 UTC m=+75.134358268"
Jul 07 11:16:33.622054 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.622001    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wxjtb" podStartSLOduration=34.171543815 podStartE2EDuration="41.621981844s" totalImagesPullingTime="7.450438029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:33.620414301 +0000 UTC m=+75.159105715" watchObservedRunningTime="2026-07-07 11:16:33.621981844 +0000 UTC m=+75.160673255"
Jul 07 11:16:33.647550 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.647455    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cxw5n" podStartSLOduration=17.765225949 podStartE2EDuration="29.64743961s" totalImagesPullingTime="11.882213661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:16:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:33.644536621 +0000 UTC m=+75.183228056" watchObservedRunningTime="2026-07-07 11:16:33.64743961 +0000 UTC m=+75.186131022"
Jul 07 11:16:33.733789 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.733736    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-rf5tn" podStartSLOduration=43.313579087 podStartE2EDuration="50.733715484s" totalImagesPullingTime="7.420136397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:33.673414599 +0000 UTC m=+75.212106011" watchObservedRunningTime="2026-07-07 11:16:33.733715484 +0000 UTC m=+75.272406897"
Jul 07 11:16:33.824480 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.824443    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:33.929758 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:33.929660    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:34.058991 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:34.058952    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:34.408394 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:34.408357    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:34.512745 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:34.512711    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:34.615352 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:34.615301    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:34.709551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:34.708262    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:34.801869 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:34.801838    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:35.124687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:35.124651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:35.204617 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:35.204584    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:35.283991 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:35.283936    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:35.393417 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:35.393291    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:35.490127 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:35.490075    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:35.749270 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:35.749095    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:35.851622 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:35.851588    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:36.232395 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:36.232364    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:36.352455 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:36.352419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:36.897828 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:36.897789    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:37.010251 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:37.010213    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:39.563733 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.563689    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:39.582007 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.581981    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j" event={"ID":"02d71c7f-00a6-489f-b769-906441000bec","Type":"ContainerStarted","Data":"d5c43ff00e2f9678bfa11e634bfc7becf94a85b736bc02c3e550b434ca302d4c"}
Jul 07 11:16:39.583376 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.583347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9" event={"ID":"a3f38aee-83a7-41ac-ba0b-680edf736cd4","Type":"ContainerStarted","Data":"ea5fdcc7e7daf99cfdcffa26b4b850500184395584758d5da107fac15f4821ac"}
Jul 07 11:16:39.583527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.583510    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:39.585520 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.585486    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9"
Jul 07 11:16:39.585813 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.585567    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b" event={"ID":"c6569d98-137a-4f0c-a398-790d08fc1b96","Type":"ContainerStarted","Data":"f23532ed7d0fe081f6d26e9c24e05ae6bc3029c614f0d3e9d3845c93803e283d"}
Jul 07 11:16:39.585813 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.585595    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b" event={"ID":"c6569d98-137a-4f0c-a398-790d08fc1b96","Type":"ContainerStarted","Data":"3c51fde9174117a9fbcee3f5211cbf299d2f9ab8f7e4c9cfd36cbd346acc9257"}
Jul 07 11:16:39.585813 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.585606    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b" event={"ID":"c6569d98-137a-4f0c-a398-790d08fc1b96","Type":"ContainerStarted","Data":"82f8ae6e660b0986bc768bf1fd6110ebb7d7dc13d84adb210020f852b0d122af"}
Jul 07 11:16:39.601049 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.601003    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cb8jb" podStartSLOduration=72.529324523 podStartE2EDuration="1m20.600985038s" totalImagesPullingTime="8.071660515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:33.735654093 +0000 UTC m=+75.274345505" watchObservedRunningTime="2026-07-07 11:16:39.600985038 +0000 UTC m=+81.139676451"
Jul 07 11:16:39.601782 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.601753    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bcb645d76-kph6j" podStartSLOduration=5.378283342 podStartE2EDuration="11.601740723s" totalImagesPullingTime="6.223457381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:39.601065573 +0000 UTC m=+81.139756984" watchObservedRunningTime="2026-07-07 11:16:39.601740723 +0000 UTC m=+81.140432134"
Jul 07 11:16:39.617352 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.617313    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbffb6b9c-gcfp9" podStartSLOduration=5.389633169 podStartE2EDuration="11.617301653s" totalImagesPullingTime="6.227668484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:39.616481011 +0000 UTC m=+81.155172429" watchObservedRunningTime="2026-07-07 11:16:39.617301653 +0000 UTC m=+81.155993064"
Jul 07 11:16:39.623365 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.623338    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:39.635690 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:39.635654    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b8dd6d9b8-hzb5b" podStartSLOduration=5.367491739 podStartE2EDuration="11.63564199s" totalImagesPullingTime="6.268150251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:39.634532808 +0000 UTC m=+81.173224221" watchObservedRunningTime="2026-07-07 11:16:39.63564199 +0000 UTC m=+81.174333402"
Jul 07 11:16:41.379177 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:41.379022    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:41.488033 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:41.487994    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:41.652913 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:41.652810    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:43.560380 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:43.560348    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wxjtb"
Jul 07 11:16:44.261172 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:44.261136    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:44.940921 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:44.940891    2582 patch_prober.go:28] interesting pod/image-registry-85d77785b6-h2klp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 07 11:16:44.941301 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:44.940941    2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-85d77785b6-h2klp" podUID="a58bd4ae-21c5-427d-a2e3-0791f820c597" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 11:16:45.084726 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:45.084698    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:45.175594 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:45.175553    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:45.658098 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:45.658056    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:45.760644 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:45.760604    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:46.502470 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:46.502444    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:16:46.765929 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:46.765839    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:46.873786 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:46.873756    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:46.994316 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:46.994284    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:47.123071 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.122991    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mklsj"]
Jul 07 11:16:47.128428 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.128409    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.131853 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.131828    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 11:16:47.132491 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.132472    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 11:16:47.132980 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.132958    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 11:16:47.133099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.132980    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hdwhd\""
Jul 07 11:16:47.133337 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.133322    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 11:16:47.144406 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.144382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:47.164671 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.164645    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2s56\" (UniqueName: \"kubernetes.io/projected/b284ab20-3f75-40b8-95a2-ddcebd7849c8-kube-api-access-w2s56\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.164776 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.164677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-wtmp\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.164776 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.164712    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.164884 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.164791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b284ab20-3f75-40b8-95a2-ddcebd7849c8-sys\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.164884 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.164856    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b284ab20-3f75-40b8-95a2-ddcebd7849c8-root\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.164979 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.164889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-textfile\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.164979 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.164917    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-tls\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.164979 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.164943    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.165127 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.164976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b284ab20-3f75-40b8-95a2-ddcebd7849c8-metrics-client-ca\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266174 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2s56\" (UniqueName: \"kubernetes.io/projected/b284ab20-3f75-40b8-95a2-ddcebd7849c8-kube-api-access-w2s56\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266312 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266185    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-wtmp\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266312 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266243    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266312 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b284ab20-3f75-40b8-95a2-ddcebd7849c8-sys\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266312 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266307    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b284ab20-3f75-40b8-95a2-ddcebd7849c8-root\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266365    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b284ab20-3f75-40b8-95a2-ddcebd7849c8-sys\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266396    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-textfile\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266402    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-wtmp\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-tls\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266447    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266478    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b284ab20-3f75-40b8-95a2-ddcebd7849c8-metrics-client-ca\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.266522 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:47.266514    2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 07 11:16:47.266857 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:16:47.266583    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-tls podName:b284ab20-3f75-40b8-95a2-ddcebd7849c8 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:47.766564492 +0000 UTC m=+89.305255884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-tls") pod "node-exporter-mklsj" (UID: "b284ab20-3f75-40b8-95a2-ddcebd7849c8") : secret "node-exporter-tls" not found
Jul 07 11:16:47.266918 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.266875    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b284ab20-3f75-40b8-95a2-ddcebd7849c8-root\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.267139 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.267118    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-textfile\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.267208 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.267163    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b284ab20-3f75-40b8-95a2-ddcebd7849c8-metrics-client-ca\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.267208 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.267182    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.269046 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.269027    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.278389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.278357    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2s56\" (UniqueName: \"kubernetes.io/projected/b284ab20-3f75-40b8-95a2-ddcebd7849c8-kube-api-access-w2s56\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.290943 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.290919    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:47.394070 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.393990    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:47.582219 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.582164    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:47.698134 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.698057    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:47.770388 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.770363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-tls\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.772623 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.772591    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b284ab20-3f75-40b8-95a2-ddcebd7849c8-node-exporter-tls\") pod \"node-exporter-mklsj\" (UID: \"b284ab20-3f75-40b8-95a2-ddcebd7849c8\") " pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:47.837093 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:47.837058    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:48.003583 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:48.003555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:48.038342 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:48.038320    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mklsj"
Jul 07 11:16:48.049680 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:16:48.049657    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb284ab20_3f75_40b8_95a2_ddcebd7849c8.slice/crio-fb64794aeedb669251ca892ab10b7dbfa3450d786a35459ef1307877f566f947 WatchSource:0}: Error finding container fb64794aeedb669251ca892ab10b7dbfa3450d786a35459ef1307877f566f947: Status 404 returned error can't find the container with id fb64794aeedb669251ca892ab10b7dbfa3450d786a35459ef1307877f566f947
Jul 07 11:16:48.094285 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:48.094169    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:48.432069 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:48.431978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:48.493802 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:48.493766    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:48.622545 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:48.622510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mklsj" event={"ID":"b284ab20-3f75-40b8-95a2-ddcebd7849c8","Type":"ContainerStarted","Data":"fb64794aeedb669251ca892ab10b7dbfa3450d786a35459ef1307877f566f947"}
Jul 07 11:16:50.067745 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:50.067697    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:50.486669 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:50.486634    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:50.589613 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:50.589574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:50.631542 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:50.631510    2582 generic.go:356] "Generic (PLEG): container finished" podID="b284ab20-3f75-40b8-95a2-ddcebd7849c8" containerID="e78c05de53c291e2b87e49488ef3b30dabcd97e2373d3db47b57f5d755bb607d" exitCode=0
Jul 07 11:16:50.631698 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:50.631597    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mklsj" event={"ID":"b284ab20-3f75-40b8-95a2-ddcebd7849c8","Type":"ContainerDied","Data":"e78c05de53c291e2b87e49488ef3b30dabcd97e2373d3db47b57f5d755bb607d"}
Jul 07 11:16:51.638068 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:51.638016    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mklsj" event={"ID":"b284ab20-3f75-40b8-95a2-ddcebd7849c8","Type":"ContainerStarted","Data":"ef2e3c5fccd0b0a9dc0fef31fe9a82c5af17b492095a0157d63f6e6f933e7600"}
Jul 07 11:16:51.638547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:51.638081    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mklsj" event={"ID":"b284ab20-3f75-40b8-95a2-ddcebd7849c8","Type":"ContainerStarted","Data":"8afb7cdee891b63674122bc00a1bbf6fc782fe731ca5913ca9688574c9d663c4"}
Jul 07 11:16:51.657131 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:51.657102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:51.668069 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:51.668030    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mklsj" podStartSLOduration=2.789391598 podStartE2EDuration="4.668017978s" totalImagesPullingTime="1.814763593s" totalInitContainerRuntime="63.862787ms" isStatefulPod=true podCreationTimestamp="2026-07-07 11:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:51.66699424 +0000 UTC m=+93.205685654" watchObservedRunningTime="2026-07-07 11:16:51.668017978 +0000 UTC m=+93.206709391"
Jul 07 11:16:51.749029 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:51.748993    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:51.915214 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:51.915114    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:52.224514 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:52.224480    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:52.325216 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:52.325169    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:52.686779 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:52.686693    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:52.771068 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:52.771043    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:53.372573 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:53.372539    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:55.033674 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:55.033646    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85d77785b6-h2klp"]
Jul 07 11:16:55.168072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:55.168042    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:55.255532 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:55.255501    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:56.117941 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:56.117904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:56.501158 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:56.501126    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-524zs"
Jul 07 11:16:56.636107 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:56.636069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:56.705956 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:56.705925    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:57.641469 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:57.641439    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:16:57.744369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:16:57.744329    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:00.840362 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:00.840328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:00.944644 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:00.944581    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:01.521426 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:01.521397    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:01.681458 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:01.681425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:01.888609 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:01.888529    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:01.980341 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:01.980300    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:03.112782 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:03.112748    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:03.262897 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:03.262865    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:03.501256 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:03.501221    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:03.675134 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:03.675092    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:03.783905 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:03.783824    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:04.832479 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:04.832434    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:04.966321 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:04.966285    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:05.119849 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:05.119753    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:05.249824 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:05.249796    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:05.356630 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:05.356597    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:10.712912 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:10.712878    2582 generic.go:356] "Generic (PLEG): container finished" podID="e8209861-6fc3-4980-8389-3cc58b1d3c40" containerID="4b881dd6d559e1ef5f72d51d7fafce391fcef6f55bc1b611d724af8defde226d" exitCode=0
Jul 07 11:17:10.713326 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:10.712956    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb" event={"ID":"e8209861-6fc3-4980-8389-3cc58b1d3c40","Type":"ContainerDied","Data":"4b881dd6d559e1ef5f72d51d7fafce391fcef6f55bc1b611d724af8defde226d"}
Jul 07 11:17:10.713326 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:10.713310    2582 scope.go:117] "RemoveContainer" containerID="4b881dd6d559e1ef5f72d51d7fafce391fcef6f55bc1b611d724af8defde226d"
Jul 07 11:17:11.718911 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:11.718878    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-cwqjb" event={"ID":"e8209861-6fc3-4980-8389-3cc58b1d3c40","Type":"ContainerStarted","Data":"9c617629c73dc0fea5ce6019489b58b0f1b1bad07024ed79ee4f3ea9bc1dd05e"}
Jul 07 11:17:20.053670 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.053607    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-85d77785b6-h2klp" podUID="a58bd4ae-21c5-427d-a2e3-0791f820c597" containerName="registry" containerID="cri-o://050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793" gracePeriod=30
Jul 07 11:17:20.074136 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.074113    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:20.143709 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.143679    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:20.282171 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.282148    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:17:20.441646 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.441574    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-trusted-ca\") pod \"a58bd4ae-21c5-427d-a2e3-0791f820c597\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") "
Jul 07 11:17:20.441646 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.441614    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-certificates\") pod \"a58bd4ae-21c5-427d-a2e3-0791f820c597\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") "
Jul 07 11:17:20.441646 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.441644    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-installation-pull-secrets\") pod \"a58bd4ae-21c5-427d-a2e3-0791f820c597\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") "
Jul 07 11:17:20.441896 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.441664    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a58bd4ae-21c5-427d-a2e3-0791f820c597-ca-trust-extracted\") pod \"a58bd4ae-21c5-427d-a2e3-0791f820c597\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") "
Jul 07 11:17:20.441896 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.441682    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67qb4\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-kube-api-access-67qb4\") pod \"a58bd4ae-21c5-427d-a2e3-0791f820c597\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") "
Jul 07 11:17:20.441896 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.441698    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-bound-sa-token\") pod \"a58bd4ae-21c5-427d-a2e3-0791f820c597\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") "
Jul 07 11:17:20.441896 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.441756    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") pod \"a58bd4ae-21c5-427d-a2e3-0791f820c597\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") "
Jul 07 11:17:20.441896 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.441825    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-image-registry-private-configuration\") pod \"a58bd4ae-21c5-427d-a2e3-0791f820c597\" (UID: \"a58bd4ae-21c5-427d-a2e3-0791f820c597\") "
Jul 07 11:17:20.442110 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.442032    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a58bd4ae-21c5-427d-a2e3-0791f820c597" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:17:20.443007 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.442976    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a58bd4ae-21c5-427d-a2e3-0791f820c597" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:17:20.444858 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.444826    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a58bd4ae-21c5-427d-a2e3-0791f820c597" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:17:20.444978 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.444853    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-kube-api-access-67qb4" (OuterVolumeSpecName: "kube-api-access-67qb4") pod "a58bd4ae-21c5-427d-a2e3-0791f820c597" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597"). InnerVolumeSpecName "kube-api-access-67qb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:17:20.445177 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.445152    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a58bd4ae-21c5-427d-a2e3-0791f820c597" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:17:20.445491 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.445448    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a58bd4ae-21c5-427d-a2e3-0791f820c597" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:17:20.445491 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.445472    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a58bd4ae-21c5-427d-a2e3-0791f820c597" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:17:20.453772 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.453742    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a58bd4ae-21c5-427d-a2e3-0791f820c597-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a58bd4ae-21c5-427d-a2e3-0791f820c597" (UID: "a58bd4ae-21c5-427d-a2e3-0791f820c597"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:17:20.542761 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.542726    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:17:20.542761 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.542759    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-image-registry-private-configuration\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:17:20.542761 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.542770    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-trusted-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:17:20.542972 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.542780    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a58bd4ae-21c5-427d-a2e3-0791f820c597-registry-certificates\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:17:20.542972 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.542789    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a58bd4ae-21c5-427d-a2e3-0791f820c597-installation-pull-secrets\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:17:20.542972 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.542798    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a58bd4ae-21c5-427d-a2e3-0791f820c597-ca-trust-extracted\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:17:20.542972 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.542806    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67qb4\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-kube-api-access-67qb4\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:17:20.542972 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.542813    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a58bd4ae-21c5-427d-a2e3-0791f820c597-bound-sa-token\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:17:20.752992 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.752958    2582 generic.go:356] "Generic (PLEG): container finished" podID="a58bd4ae-21c5-427d-a2e3-0791f820c597" containerID="050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793" exitCode=0
Jul 07 11:17:20.753226 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.752999    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85d77785b6-h2klp"
Jul 07 11:17:20.753226 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.753011    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85d77785b6-h2klp" event={"ID":"a58bd4ae-21c5-427d-a2e3-0791f820c597","Type":"ContainerDied","Data":"050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793"}
Jul 07 11:17:20.753226 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.753059    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85d77785b6-h2klp" event={"ID":"a58bd4ae-21c5-427d-a2e3-0791f820c597","Type":"ContainerDied","Data":"a4e57e44060267be5e447a55492d920ab5d91e0aeb39d8f277aaa9d26ca30a9a"}
Jul 07 11:17:20.753226 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.753084    2582 scope.go:117] "RemoveContainer" containerID="050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793"
Jul 07 11:17:20.761899 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.761881    2582 scope.go:117] "RemoveContainer" containerID="050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793"
Jul 07 11:17:20.762166 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:17:20.762145    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793\": container with ID starting with 050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793 not found: ID does not exist" containerID="050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793"
Jul 07 11:17:20.762252 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.762178    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793"} err="failed to get container status \"050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793\": rpc error: code = NotFound desc = could not find container \"050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793\": container with ID starting with 050e67d68a84b6ff2cabc32aac852de85690fb78f8d4eecdb06d0c9c1ac88793 not found: ID does not exist"
Jul 07 11:17:20.775634 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.775614    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85d77785b6-h2klp"]
Jul 07 11:17:20.780148 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:20.780125    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-85d77785b6-h2klp"]
Jul 07 11:17:21.024747 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:21.024675    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a58bd4ae-21c5-427d-a2e3-0791f820c597" path="/var/lib/kubelet/pods/a58bd4ae-21c5-427d-a2e3-0791f820c597/volumes"
Jul 07 11:17:34.809303 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:34.809272    2582 generic.go:356] "Generic (PLEG): container finished" podID="c98ff6c7-186a-42af-af1e-ba05d65dba1a" containerID="0be06590e2a3d61f26a25442ce0e39ce23cef65b35ba4f8fe2e1d60cb9b4428c" exitCode=0
Jul 07 11:17:34.809768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:34.809346    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rztt6" event={"ID":"c98ff6c7-186a-42af-af1e-ba05d65dba1a","Type":"ContainerDied","Data":"0be06590e2a3d61f26a25442ce0e39ce23cef65b35ba4f8fe2e1d60cb9b4428c"}
Jul 07 11:17:34.809768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:34.809697    2582 scope.go:117] "RemoveContainer" containerID="0be06590e2a3d61f26a25442ce0e39ce23cef65b35ba4f8fe2e1d60cb9b4428c"
Jul 07 11:17:34.810797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:34.810771    2582 generic.go:356] "Generic (PLEG): container finished" podID="40778e43-f44b-409e-8ed5-23f4971b8407" containerID="d26394751a76f3ca37ed47f7880bfda90536266d5bb346023234a4fac50763be" exitCode=0
Jul 07 11:17:34.810905 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:34.810799    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl" event={"ID":"40778e43-f44b-409e-8ed5-23f4971b8407","Type":"ContainerDied","Data":"d26394751a76f3ca37ed47f7880bfda90536266d5bb346023234a4fac50763be"}
Jul 07 11:17:34.811120 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:34.811105    2582 scope.go:117] "RemoveContainer" containerID="d26394751a76f3ca37ed47f7880bfda90536266d5bb346023234a4fac50763be"
Jul 07 11:17:35.817235 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:35.817185    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8xtcl" event={"ID":"40778e43-f44b-409e-8ed5-23f4971b8407","Type":"ContainerStarted","Data":"636779ec0824d670673e5f56548a759fab1573e2c8061d0634bc9bd88695aa34"}
Jul 07 11:17:35.819051 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:35.819025    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-rztt6" event={"ID":"c98ff6c7-186a-42af-af1e-ba05d65dba1a","Type":"ContainerStarted","Data":"88e225eb4cf68ba29f3a39e25595c465e9e3278c20c614f05f6876fc7c1624d9"}
Jul 07 11:17:36.355653 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:36.355611    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:36.685436 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:36.685359    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:45.266613 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:45.266575    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:46.113352 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:46.113320    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:46.176055 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:46.176022    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:46.252914 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:46.252881    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:46.314278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:46.314241    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:46.892219 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:46.892146    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:46.984252 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:46.984130    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:57.070265 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:57.070232    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:57.171265 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:57.171226    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:17:57.227230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:17:57.227189    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:18:51.707881 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:18:51.707846    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:18:51.796101 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:18:51.796069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:03.613942 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:03.613904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:03.672386 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:03.672359    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:05.621226 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:05.621147    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:05.686750 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:05.686718    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:08.319488 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:08.319454    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:08.385851 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:08.385817    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:08.458108 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:08.458075    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:08.527338 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:08.527310    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:09.051667 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:09.051638    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:09.123365 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:09.123308    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:19.365869 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:19.365837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:19.466439 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:19.466410    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:19.527161 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:19.527135    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:20.059661 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:20.059613    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:20.158831 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:20.158798    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:20.215031 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:20.215003    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:41.205835 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:41.205795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:19:41.412421 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:19:41.412343    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:08.586941 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:08.586906    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:09.517370 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:09.517338    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:12.206318 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:12.206279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:12.306975 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:12.306938    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:13.347140 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:13.347105    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:13.453315 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:13.453283    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:14.424236 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:14.424178    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:14.688738 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:14.688661    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:16.808284 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:16.808255    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:17.005149 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:17.005123    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:18.880703 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:18.880673    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:20:18.891937 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:18.891914    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:20:18.897517 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:18.897498    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 11:20:18.991850 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:18.991823    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:19.056796 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:19.056758    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:24.487080 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:24.487038    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:25.203938 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:25.203899    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:25.311560 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:25.311533    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:30.715130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:30.715090    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:30.816994 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:30.816964    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:32.540449 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:32.540412    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:38.273056 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:38.273023    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:50.114606 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:50.114566    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:50.221076 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:50.221045    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:50.777151 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:50.777118    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:51.086310 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.086226    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:51.194772 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.194738    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:51.234109 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.234077    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"]
Jul 07 11:20:51.234399 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.234386    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a58bd4ae-21c5-427d-a2e3-0791f820c597" containerName="registry"
Jul 07 11:20:51.234467 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.234401    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a58bd4ae-21c5-427d-a2e3-0791f820c597" containerName="registry"
Jul 07 11:20:51.234503 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.234469    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a58bd4ae-21c5-427d-a2e3-0791f820c597" containerName="registry"
Jul 07 11:20:51.237451 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.237432    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.241717 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.241696    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 11:20:51.241840 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.241792    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 11:20:51.241840 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.241818    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tp44d\""
Jul 07 11:20:51.241959 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.241894    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 11:20:51.241959 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.241921    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 11:20:51.305497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.305472    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"]
Jul 07 11:20:51.352984 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.352890    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:51.369998 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.369966    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b2d0fdb-d643-45d2-99bd-851a456b14fa-tmp\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.370148 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.370017    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5b2d0fdb-d643-45d2-99bd-851a456b14fa-manifests\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.370148 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.370046    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b2d0fdb-d643-45d2-99bd-851a456b14fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.370148 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.370074    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqhkg\" (UniqueName: \"kubernetes.io/projected/5b2d0fdb-d643-45d2-99bd-851a456b14fa-kube-api-access-mqhkg\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.370327 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.370167    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b2d0fdb-d643-45d2-99bd-851a456b14fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.454263 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.454229    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:51.471252 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.471223    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b2d0fdb-d643-45d2-99bd-851a456b14fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.471374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.471275    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b2d0fdb-d643-45d2-99bd-851a456b14fa-tmp\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.471374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.471307    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5b2d0fdb-d643-45d2-99bd-851a456b14fa-manifests\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.471374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.471326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b2d0fdb-d643-45d2-99bd-851a456b14fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.471374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.471348    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqhkg\" (UniqueName: \"kubernetes.io/projected/5b2d0fdb-d643-45d2-99bd-851a456b14fa-kube-api-access-mqhkg\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.471772 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.471752    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5b2d0fdb-d643-45d2-99bd-851a456b14fa-manifests\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.473665 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.473643    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b2d0fdb-d643-45d2-99bd-851a456b14fa-tmp\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.473857 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.473837    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b2d0fdb-d643-45d2-99bd-851a456b14fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.473900 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.473866    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b2d0fdb-d643-45d2-99bd-851a456b14fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.481555 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.481525    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqhkg\" (UniqueName: \"kubernetes.io/projected/5b2d0fdb-d643-45d2-99bd-851a456b14fa-kube-api-access-mqhkg\") pod \"opendatahub-operator-controller-manager-59ddd9ddd8-4kn54\" (UID: \"5b2d0fdb-d643-45d2-99bd-851a456b14fa\") " pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.548438 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.548410    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:51.553318 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.553286    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:51.659441 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.659408    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:51.689762 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.689674    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"]
Jul 07 11:20:51.692474 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:20:51.692360    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2d0fdb_d643_45d2_99bd_851a456b14fa.slice/crio-9e7ce57a023236ed13d8feba93408a756e3536fe94c8c089ff777a80d99efd8d WatchSource:0}: Error finding container 9e7ce57a023236ed13d8feba93408a756e3536fe94c8c089ff777a80d99efd8d: Status 404 returned error can't find the container with id 9e7ce57a023236ed13d8feba93408a756e3536fe94c8c089ff777a80d99efd8d
Jul 07 11:20:51.694184 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.694167    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:20:51.770051 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.770016    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:51.869449 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.869417    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:51.961206 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:51.961106    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:52.329363 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:52.329328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:52.545423 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:52.545388    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54" event={"ID":"5b2d0fdb-d643-45d2-99bd-851a456b14fa","Type":"ContainerStarted","Data":"9e7ce57a023236ed13d8feba93408a756e3536fe94c8c089ff777a80d99efd8d"}
Jul 07 11:20:53.071475 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:53.071325    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:53.170283 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:53.170244    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:55.566969 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:55.566930    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54" event={"ID":"5b2d0fdb-d643-45d2-99bd-851a456b14fa","Type":"ContainerStarted","Data":"97cba01b258d21f61af02808ff97700d4f358137354eeb57715aa6f802f72b61"}
Jul 07 11:20:56.572370 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:56.572284    2582 generic.go:356] "Generic (PLEG): container finished" podID="5b2d0fdb-d643-45d2-99bd-851a456b14fa" containerID="97cba01b258d21f61af02808ff97700d4f358137354eeb57715aa6f802f72b61" exitCode=0
Jul 07 11:20:56.572370 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:56.572346    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54" event={"ID":"5b2d0fdb-d643-45d2-99bd-851a456b14fa","Type":"ContainerDied","Data":"97cba01b258d21f61af02808ff97700d4f358137354eeb57715aa6f802f72b61"}
Jul 07 11:20:57.578507 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:57.578465    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54" event={"ID":"5b2d0fdb-d643-45d2-99bd-851a456b14fa","Type":"ContainerStarted","Data":"d047148a402909bb9aa54e56d3e4fb3e5e1eb3605a944fd3d1212c2b896eb8d1"}
Jul 07 11:20:57.578852 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:57.578597    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:20:57.610692 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:57.610644    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54" podStartSLOduration=2.364815422 podStartE2EDuration="6.610630498s" totalImagesPullingTime="3.221497286s" totalInitContainerRuntime="1.02431779s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:57.608307186 +0000 UTC m=+339.146998598" watchObservedRunningTime="2026-07-07 11:20:57.610630498 +0000 UTC m=+339.149321909"
Jul 07 11:20:58.781651 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:58.781618    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:20:58.879652 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:20:58.879617    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:08.584754 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:08.584723    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59ddd9ddd8-4kn54"
Jul 07 11:21:10.211567 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:10.211538    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:10.357160 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:10.357123    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:10.665645 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:10.665608    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:10.922818 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:10.922727    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:11.117592 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:11.117559    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:11.215934 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:11.215898    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:11.319165 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:11.319091    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:11.593184 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:11.593093    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:12.036716 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:12.036673    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:12.348483 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:12.348404    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:12.448100 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:12.448071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:12.548629 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:12.548568    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:13.512805 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:13.512773    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:18.367548 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:18.367516    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:18.480016 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:18.479979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:18.706215 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:18.706124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:18.928697 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:18.928667    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:19.048996 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:19.048959    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:21.506248 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.506211    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"]
Jul 07 11:21:21.511036 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.511016    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.517259 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.517094    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 11:21:21.517259 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.517125    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 11:21:21.517259 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.517130    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 11:21:21.517259 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.517170    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 11:21:21.517967 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.517633    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:21:21.517967 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.517745    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k5prt\""
Jul 07 11:21:21.534300 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.534274    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"]
Jul 07 11:21:21.605208 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.605169    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsqzj\" (UniqueName: \"kubernetes.io/projected/27031715-9c2d-4071-8e38-f3b1af1c6775-kube-api-access-xsqzj\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.605348 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.605223    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27031715-9c2d-4071-8e38-f3b1af1c6775-cert\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.605348 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.605302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/27031715-9c2d-4071-8e38-f3b1af1c6775-manager-config\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.605348 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.605325    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/27031715-9c2d-4071-8e38-f3b1af1c6775-metrics-cert\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.706350 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.706323    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/27031715-9c2d-4071-8e38-f3b1af1c6775-manager-config\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.706499 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.706359    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/27031715-9c2d-4071-8e38-f3b1af1c6775-metrics-cert\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.706499 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.706450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsqzj\" (UniqueName: \"kubernetes.io/projected/27031715-9c2d-4071-8e38-f3b1af1c6775-kube-api-access-xsqzj\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.706499 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.706475    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27031715-9c2d-4071-8e38-f3b1af1c6775-cert\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.707044 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.707024    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/27031715-9c2d-4071-8e38-f3b1af1c6775-manager-config\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.708791 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.708772    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/27031715-9c2d-4071-8e38-f3b1af1c6775-metrics-cert\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.708944 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.708924    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27031715-9c2d-4071-8e38-f3b1af1c6775-cert\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.720353 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.720333    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsqzj\" (UniqueName: \"kubernetes.io/projected/27031715-9c2d-4071-8e38-f3b1af1c6775-kube-api-access-xsqzj\") pod \"lws-controller-manager-65f8ccd9c5-b7jq2\" (UID: \"27031715-9c2d-4071-8e38-f3b1af1c6775\") " pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.820520 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.820444    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:21.985375 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:21.985347    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"]
Jul 07 11:21:21.989347 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:21:21.989319    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27031715_9c2d_4071_8e38_f3b1af1c6775.slice/crio-eb52686e3a3b516485b5b82648399c4e7bb219b1d47e09bfcb64899597fe66f3 WatchSource:0}: Error finding container eb52686e3a3b516485b5b82648399c4e7bb219b1d47e09bfcb64899597fe66f3: Status 404 returned error can't find the container with id eb52686e3a3b516485b5b82648399c4e7bb219b1d47e09bfcb64899597fe66f3
Jul 07 11:21:22.681831 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:22.681790    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2" event={"ID":"27031715-9c2d-4071-8e38-f3b1af1c6775","Type":"ContainerStarted","Data":"eb52686e3a3b516485b5b82648399c4e7bb219b1d47e09bfcb64899597fe66f3"}
Jul 07 11:21:22.921665 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:22.921589    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:23.071034 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:23.071004    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:23.666416 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:23.666382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:23.872275 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:23.872225    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:24.334361 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:24.334312    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:24.474630 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:24.474583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:25.696222 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:25.696168    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2" event={"ID":"27031715-9c2d-4071-8e38-f3b1af1c6775","Type":"ContainerStarted","Data":"82b76e3a5735b81a9ad470a7d3682b82bf0f197d7f669e414e2f2d609ad1ecf4"}
Jul 07 11:21:25.696695 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:25.696272    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:25.715847 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:25.715797    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2" podStartSLOduration=2.066089035 podStartE2EDuration="4.715783296s" totalImagesPullingTime="2.649694261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:25.714080338 +0000 UTC m=+367.252771740" watchObservedRunningTime="2026-07-07 11:21:25.715783296 +0000 UTC m=+367.254474758"
Jul 07 11:21:34.462916 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:34.462882    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:36.703735 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:36.703701    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65f8ccd9c5-b7jq2"
Jul 07 11:21:51.203376 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:51.203308    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:21:51.256150 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:21:51.256123    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:11.642449 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:11.642406    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:11.742632 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:11.742586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:11.857593 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:11.857554    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:12.582071 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:12.582033    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:12.788841 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:12.788808    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:41.325410 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:41.325331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:44.968540 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:44.968504    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:45.073047 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:45.073008    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:49.330171 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:49.330137    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:49.740522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:49.740486    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:51.175400 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:51.175367    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:52.746335 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:52.746286    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:53.587214 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:53.587160    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:53.698396 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:53.698364    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:54.067682 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:54.067651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:54.166286 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:54.166255    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:54.275649 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:54.275615    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:54.782274 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:54.782241    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:54.883421 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:54.883382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:55.018000 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.017968    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh"]
Jul 07 11:22:55.021235 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.021214    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh"
Jul 07 11:22:55.024129 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.024109    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-q5tjc\""
Jul 07 11:22:55.024320 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.024298    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 11:22:55.025259 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.025235    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 07 11:22:55.025365 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.025320    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 11:22:55.036145 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.036094    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh"]
Jul 07 11:22:55.137005 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.136973    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkznd\" (UniqueName: \"kubernetes.io/projected/12774182-f4f5-46ca-93cb-96f04a2118e8-kube-api-access-bkznd\") pod \"dns-operator-controller-manager-864dd78c4c-crjgh\" (UID: \"12774182-f4f5-46ca-93cb-96f04a2118e8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh"
Jul 07 11:22:55.237343 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.237309    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkznd\" (UniqueName: \"kubernetes.io/projected/12774182-f4f5-46ca-93cb-96f04a2118e8-kube-api-access-bkznd\") pod \"dns-operator-controller-manager-864dd78c4c-crjgh\" (UID: \"12774182-f4f5-46ca-93cb-96f04a2118e8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh"
Jul 07 11:22:55.248987 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.248961    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkznd\" (UniqueName: \"kubernetes.io/projected/12774182-f4f5-46ca-93cb-96f04a2118e8-kube-api-access-bkznd\") pod \"dns-operator-controller-manager-864dd78c4c-crjgh\" (UID: \"12774182-f4f5-46ca-93cb-96f04a2118e8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh"
Jul 07 11:22:55.332619 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.332535    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh"
Jul 07 11:22:55.459149 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.459116    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh"]
Jul 07 11:22:55.462261 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:22:55.462234    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12774182_f4f5_46ca_93cb_96f04a2118e8.slice/crio-9606bfe003854e40090756ecf8f5bdc1697047fc149b7f70969c63d1919a6f73 WatchSource:0}: Error finding container 9606bfe003854e40090756ecf8f5bdc1697047fc149b7f70969c63d1919a6f73: Status 404 returned error can't find the container with id 9606bfe003854e40090756ecf8f5bdc1697047fc149b7f70969c63d1919a6f73
Jul 07 11:22:55.581936 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:55.581903    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:56.074897 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:56.074867    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh" event={"ID":"12774182-f4f5-46ca-93cb-96f04a2118e8","Type":"ContainerStarted","Data":"9606bfe003854e40090756ecf8f5bdc1697047fc149b7f70969c63d1919a6f73"}
Jul 07 11:22:56.931818 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:56.931760    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:57.031117 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:57.031089    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:58.084490 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.084455    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh" event={"ID":"12774182-f4f5-46ca-93cb-96f04a2118e8","Type":"ContainerStarted","Data":"ba763a2daf9fb0991a46740fc3b1c30f2a3f59e3a8d36f2415dc0ddda7a412a2"}
Jul 07 11:22:58.084854 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.084570    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh"
Jul 07 11:22:58.109332 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.109282    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh" podStartSLOduration=1.972518033 podStartE2EDuration="4.109266358s" totalImagesPullingTime="2.136748325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:22:58.108600369 +0000 UTC m=+459.647291805" watchObservedRunningTime="2026-07-07 11:22:58.109266358 +0000 UTC m=+459.647957769"
Jul 07 11:22:58.529296 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.529264    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"]
Jul 07 11:22:58.532562 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.532540    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:58.535168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.535091    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-w7m48\""
Jul 07 11:22:58.535168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.535127    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 07 11:22:58.535168 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.535135    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 07 11:22:58.550949 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.550928    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"]
Jul 07 11:22:58.552630 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.552605    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:58.668255 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.668219    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgb9b\" (UniqueName: \"kubernetes.io/projected/cba36d43-9721-40af-a1f1-73793d352705-kube-api-access-qgb9b\") pod \"kuadrant-console-plugin-66b5774985-6tpdf\" (UID: \"cba36d43-9721-40af-a1f1-73793d352705\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:58.668378 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.668282    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cba36d43-9721-40af-a1f1-73793d352705-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6tpdf\" (UID: \"cba36d43-9721-40af-a1f1-73793d352705\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:58.668434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.668370    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cba36d43-9721-40af-a1f1-73793d352705-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6tpdf\" (UID: \"cba36d43-9721-40af-a1f1-73793d352705\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:58.769750 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.769719    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgb9b\" (UniqueName: \"kubernetes.io/projected/cba36d43-9721-40af-a1f1-73793d352705-kube-api-access-qgb9b\") pod \"kuadrant-console-plugin-66b5774985-6tpdf\" (UID: \"cba36d43-9721-40af-a1f1-73793d352705\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:58.769902 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.769767    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cba36d43-9721-40af-a1f1-73793d352705-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6tpdf\" (UID: \"cba36d43-9721-40af-a1f1-73793d352705\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:58.769902 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.769802    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cba36d43-9721-40af-a1f1-73793d352705-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6tpdf\" (UID: \"cba36d43-9721-40af-a1f1-73793d352705\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:58.770008 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:22:58.769929    2582 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 07 11:22:58.770008 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:22:58.769997    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cba36d43-9721-40af-a1f1-73793d352705-plugin-serving-cert podName:cba36d43-9721-40af-a1f1-73793d352705 nodeName:}" failed. No retries permitted until 2026-07-07 11:22:59.269981331 +0000 UTC m=+460.808672721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/cba36d43-9721-40af-a1f1-73793d352705-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-6tpdf" (UID: "cba36d43-9721-40af-a1f1-73793d352705") : secret "plugin-serving-cert" not found
Jul 07 11:22:58.770399 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.770382    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cba36d43-9721-40af-a1f1-73793d352705-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6tpdf\" (UID: \"cba36d43-9721-40af-a1f1-73793d352705\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:58.781324 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.781267    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgb9b\" (UniqueName: \"kubernetes.io/projected/cba36d43-9721-40af-a1f1-73793d352705-kube-api-access-qgb9b\") pod \"kuadrant-console-plugin-66b5774985-6tpdf\" (UID: \"cba36d43-9721-40af-a1f1-73793d352705\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:58.929962 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:58.929930    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:22:59.274521 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:59.274482    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cba36d43-9721-40af-a1f1-73793d352705-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6tpdf\" (UID: \"cba36d43-9721-40af-a1f1-73793d352705\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:59.276997 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:59.276965    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cba36d43-9721-40af-a1f1-73793d352705-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6tpdf\" (UID: \"cba36d43-9721-40af-a1f1-73793d352705\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:59.443473 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:59.443440    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"
Jul 07 11:22:59.570976 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:22:59.570947    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf"]
Jul 07 11:22:59.572986 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:22:59.572953    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcba36d43_9721_40af_a1f1_73793d352705.slice/crio-daa40a34a368e7dc68a40a5584cd430fcc8d889f13251b9fa11715ab7a885614 WatchSource:0}: Error finding container daa40a34a368e7dc68a40a5584cd430fcc8d889f13251b9fa11715ab7a885614: Status 404 returned error can't find the container with id daa40a34a368e7dc68a40a5584cd430fcc8d889f13251b9fa11715ab7a885614
Jul 07 11:23:00.094381 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:00.094345    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf" event={"ID":"cba36d43-9721-40af-a1f1-73793d352705","Type":"ContainerStarted","Data":"daa40a34a368e7dc68a40a5584cd430fcc8d889f13251b9fa11715ab7a885614"}
Jul 07 11:23:00.389599 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:00.389499    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:00.457505 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:00.457448    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:05.386023 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:05.385979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:05.487040 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:05.487005    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:05.596278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:05.596245    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:09.069822 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:09.069776    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:09.090520 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:09.090456    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-crjgh"
Jul 07 11:23:09.213071 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:09.213008    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:11.491142 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.491109    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:11.607585 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.607528    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:11.721293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.721260    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"]
Jul 07 11:23:11.782635 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.782548    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"]
Jul 07 11:23:11.782635 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.782573    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:23:11.789731 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.789708    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d8l4g\""
Jul 07 11:23:11.888804 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.888757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdd8cab2-fdf7-4b35-a793-62cc853623c5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rp29x\" (UID: \"fdd8cab2-fdf7-4b35-a793-62cc853623c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:23:11.888987 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.888831    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9s6q\" (UniqueName: \"kubernetes.io/projected/fdd8cab2-fdf7-4b35-a793-62cc853623c5-kube-api-access-t9s6q\") pod \"kuadrant-operator-controller-manager-86878c769f-rp29x\" (UID: \"fdd8cab2-fdf7-4b35-a793-62cc853623c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:23:11.990222 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.990169    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9s6q\" (UniqueName: \"kubernetes.io/projected/fdd8cab2-fdf7-4b35-a793-62cc853623c5-kube-api-access-t9s6q\") pod \"kuadrant-operator-controller-manager-86878c769f-rp29x\" (UID: \"fdd8cab2-fdf7-4b35-a793-62cc853623c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:23:11.990402 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.990282    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdd8cab2-fdf7-4b35-a793-62cc853623c5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rp29x\" (UID: \"fdd8cab2-fdf7-4b35-a793-62cc853623c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:23:11.990624 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:11.990605    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdd8cab2-fdf7-4b35-a793-62cc853623c5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rp29x\" (UID: \"fdd8cab2-fdf7-4b35-a793-62cc853623c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:23:12.002337 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:12.002312    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9s6q\" (UniqueName: \"kubernetes.io/projected/fdd8cab2-fdf7-4b35-a793-62cc853623c5-kube-api-access-t9s6q\") pod \"kuadrant-operator-controller-manager-86878c769f-rp29x\" (UID: \"fdd8cab2-fdf7-4b35-a793-62cc853623c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:23:12.094266 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:12.094162    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:23:15.714460 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:15.714418    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:15.815687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:15.815640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:17.964971 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:17.964929    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:18.062188 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:18.062146    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:21.781323 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:21.781293    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"]
Jul 07 11:23:21.782832 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:23:21.782808    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd8cab2_fdf7_4b35_a793_62cc853623c5.slice/crio-878bdd913bc2a2f6e1ac22eed05038e2c382dac729d48d7c1ddf0bf9fb8f833e WatchSource:0}: Error finding container 878bdd913bc2a2f6e1ac22eed05038e2c382dac729d48d7c1ddf0bf9fb8f833e: Status 404 returned error can't find the container with id 878bdd913bc2a2f6e1ac22eed05038e2c382dac729d48d7c1ddf0bf9fb8f833e
Jul 07 11:23:22.205812 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:22.205774    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf" event={"ID":"cba36d43-9721-40af-a1f1-73793d352705","Type":"ContainerStarted","Data":"9f1e0fbdb9d93501b171cdda96bce678a1d5b52474ac0794859362494a659594"}
Jul 07 11:23:22.207109 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:22.207080    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x" event={"ID":"fdd8cab2-fdf7-4b35-a793-62cc853623c5","Type":"ContainerStarted","Data":"878bdd913bc2a2f6e1ac22eed05038e2c382dac729d48d7c1ddf0bf9fb8f833e"}
Jul 07 11:23:22.226158 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:22.226112    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6tpdf" podStartSLOduration=2.111444704 podStartE2EDuration="24.22609914s" totalImagesPullingTime="22.114654436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:23:22.224897857 +0000 UTC m=+483.763589292" watchObservedRunningTime="2026-07-07 11:23:22.22609914 +0000 UTC m=+483.764790552"
Jul 07 11:23:26.748605 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:26.748566    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:26.852488 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:26.852454    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:29.231776 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:29.231739    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:29.241283 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:29.241253    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x" event={"ID":"fdd8cab2-fdf7-4b35-a793-62cc853623c5","Type":"ContainerStarted","Data":"0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35"}
Jul 07 11:23:29.241450 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:29.241352    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:23:29.265603 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:29.265408    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x" podStartSLOduration=11.390698104 podStartE2EDuration="18.265388307s" totalImagesPullingTime="6.874690203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:23:29.263542923 +0000 UTC m=+490.802234339" watchObservedRunningTime="2026-07-07 11:23:29.265388307 +0000 UTC m=+490.804079720"
Jul 07 11:23:29.328116 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:29.328083    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:33.589502 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:33.589470    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:33.693552 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:33.693519    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:36.812067 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:36.812011    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:36.918694 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:36.918667    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:40.247581 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:40.247551    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:23:46.166533 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:46.166493    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:46.266601 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:46.266570    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:48.600656 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:48.600622    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:48.700679 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:48.700654    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:56.662372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:56.662279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:23:56.958106 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:23:56.958016    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:01.164106 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.164073    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:01.268805 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.268771    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:01.371097 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.371061    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:01.589455 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.589424    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:24:01.639884 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.639849    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:24:01.639884 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.639883    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:24:01.640076 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.639971    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"
Jul 07 11:24:01.642664 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.642645    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 11:24:01.809156 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.809129    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7bfbf03-4f84-4b4a-9009-b8f62a653b65-config-file\") pod \"limitador-limitador-54d6d988d5-2lhcn\" (UID: \"f7bfbf03-4f84-4b4a-9009-b8f62a653b65\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"
Jul 07 11:24:01.809156 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.809159    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cg62\" (UniqueName: \"kubernetes.io/projected/f7bfbf03-4f84-4b4a-9009-b8f62a653b65-kube-api-access-4cg62\") pod \"limitador-limitador-54d6d988d5-2lhcn\" (UID: \"f7bfbf03-4f84-4b4a-9009-b8f62a653b65\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"
Jul 07 11:24:01.909853 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.909782    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cg62\" (UniqueName: \"kubernetes.io/projected/f7bfbf03-4f84-4b4a-9009-b8f62a653b65-kube-api-access-4cg62\") pod \"limitador-limitador-54d6d988d5-2lhcn\" (UID: \"f7bfbf03-4f84-4b4a-9009-b8f62a653b65\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"
Jul 07 11:24:01.909972 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.909865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7bfbf03-4f84-4b4a-9009-b8f62a653b65-config-file\") pod \"limitador-limitador-54d6d988d5-2lhcn\" (UID: \"f7bfbf03-4f84-4b4a-9009-b8f62a653b65\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"
Jul 07 11:24:01.910467 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.910449    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7bfbf03-4f84-4b4a-9009-b8f62a653b65-config-file\") pod \"limitador-limitador-54d6d988d5-2lhcn\" (UID: \"f7bfbf03-4f84-4b4a-9009-b8f62a653b65\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"
Jul 07 11:24:01.919573 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.919555    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cg62\" (UniqueName: \"kubernetes.io/projected/f7bfbf03-4f84-4b4a-9009-b8f62a653b65-kube-api-access-4cg62\") pod \"limitador-limitador-54d6d988d5-2lhcn\" (UID: \"f7bfbf03-4f84-4b4a-9009-b8f62a653b65\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"
Jul 07 11:24:01.950563 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:01.950540    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"
Jul 07 11:24:02.050170 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:02.050139    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:02.082896 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:02.082867    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:24:02.085732 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:24:02.085707    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7bfbf03_4f84_4b4a_9009_b8f62a653b65.slice/crio-50170db575b96df432d18989b9219a22fb861da80effed1961e7199c54c6c084 WatchSource:0}: Error finding container 50170db575b96df432d18989b9219a22fb861da80effed1961e7199c54c6c084: Status 404 returned error can't find the container with id 50170db575b96df432d18989b9219a22fb861da80effed1961e7199c54c6c084
Jul 07 11:24:02.120478 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:02.120454    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:02.383923 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:02.383891    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn" event={"ID":"f7bfbf03-4f84-4b4a-9009-b8f62a653b65","Type":"ContainerStarted","Data":"50170db575b96df432d18989b9219a22fb861da80effed1961e7199c54c6c084"}
Jul 07 11:24:05.400578 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:05.400539    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn" event={"ID":"f7bfbf03-4f84-4b4a-9009-b8f62a653b65","Type":"ContainerStarted","Data":"e0e6bace74f6beba95fac7776a9b8b3cbb129f42d6f188a344868f0cf6c4920f"}
Jul 07 11:24:05.400971 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:05.400828    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"
Jul 07 11:24:05.422200 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:05.422157    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn" podStartSLOduration=1.84765359 podStartE2EDuration="4.422143359s" totalImagesPullingTime="2.574489769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:05.419860585 +0000 UTC m=+526.958551997" watchObservedRunningTime="2026-07-07 11:24:05.422143359 +0000 UTC m=+526.960834770"
Jul 07 11:24:16.406499 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:16.406429    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"
Jul 07 11:24:22.734036 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:22.734000    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:34.665989 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:34.665943    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:34.792292 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:34.792262    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:40.623076 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:40.623039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:40.720059 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:40.720026    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:40.780384 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:40.780345    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:41.207289 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:41.207247    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:44.273012 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:44.272980    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:24:44.392730 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:24:44.392696    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:04.951643 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:04.951607    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:17.527190 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:17.527151    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:18.917346 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:18.917317    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:25:18.919560 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:18.919534    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:25:18.964897 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:18.964863    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:25:18.986897 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:18.986860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:27.625427 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:27.625392    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:27.757831 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:27.757794    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:37.413960 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:37.413915    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:37.513799 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:37.513766    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:40.599299 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:40.599260    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:40.700376 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:40.700330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:47.404668 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:47.404572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:47.839775 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:47.839744    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:25:51.639983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:51.639947    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:25:52.229953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:25:52.229917    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:26:05.216800 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:05.216766    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:26:19.405621 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.405590    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"]
Jul 07 11:26:19.408895 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.408877    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.412627 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.412601    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 07 11:26:19.412748 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.412633    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 11:26:19.412748 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.412636    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5gs4g\""
Jul 07 11:26:19.412748 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.412688    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 11:26:19.418361 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.418341    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"]
Jul 07 11:26:19.496392 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.496363    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:26:19.509415 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.509391    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.509527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.509424    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.509527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.509445    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.509527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.509468    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdg27\" (UniqueName: \"kubernetes.io/projected/9ca7d180-7e34-455b-940c-6a611f6a7d66-kube-api-access-fdg27\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.509527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.509504    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.509527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.509520    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ca7d180-7e34-455b-940c-6a611f6a7d66-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.609861 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.609828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.610014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.609863    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ca7d180-7e34-455b-940c-6a611f6a7d66-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.610014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.609912    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.610014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.609945    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.610014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.609973    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.610014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.610010    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdg27\" (UniqueName: \"kubernetes.io/projected/9ca7d180-7e34-455b-940c-6a611f6a7d66-kube-api-access-fdg27\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.610321 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.610298    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.610381 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.610356    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.610436 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.610411    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.612118 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.612099    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ca7d180-7e34-455b-940c-6a611f6a7d66-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.612497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.612478    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ca7d180-7e34-455b-940c-6a611f6a7d66-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.618611 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.618593    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdg27\" (UniqueName: \"kubernetes.io/projected/9ca7d180-7e34-455b-940c-6a611f6a7d66-kube-api-access-fdg27\") pod \"e2e-trlp-test-simulated-kserve-598466548d-d8wfv\" (UID: \"9ca7d180-7e34-455b-940c-6a611f6a7d66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.719277 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.719256    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:19.849412 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.849384    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"]
Jul 07 11:26:19.851512 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:26:19.851475    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca7d180_7e34_455b_940c_6a611f6a7d66.slice/crio-d2ec10a9ae3f5bc8571a25ce42d36c878010002965b2551efa409c2083930c09 WatchSource:0}: Error finding container d2ec10a9ae3f5bc8571a25ce42d36c878010002965b2551efa409c2083930c09: Status 404 returned error can't find the container with id d2ec10a9ae3f5bc8571a25ce42d36c878010002965b2551efa409c2083930c09
Jul 07 11:26:19.852750 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.852732    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:26:19.977039 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:19.976966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv" event={"ID":"9ca7d180-7e34-455b-940c-6a611f6a7d66","Type":"ContainerStarted","Data":"d2ec10a9ae3f5bc8571a25ce42d36c878010002965b2551efa409c2083930c09"}
Jul 07 11:26:20.723001 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:20.722967    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:26:21.987287 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:21.987256    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv" event={"ID":"9ca7d180-7e34-455b-940c-6a611f6a7d66","Type":"ContainerStarted","Data":"ec7da7f8afa77ce1e89771d5e3f3806b38b84bf7fe218b1b5549a8f6ec1b6655"}
Jul 07 11:26:22.008964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:22.008898    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv" podStartSLOduration=1.880339256 podStartE2EDuration="3.008881614s" totalImagesPullingTime="1.128542358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:26:22.008226308 +0000 UTC m=+663.546917712" watchObservedRunningTime="2026-07-07 11:26:22.008881614 +0000 UTC m=+663.547573034"
Jul 07 11:26:29.720354 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:29.720323    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:29.720810 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:29.720386    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:29.732337 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:29.732315    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:30.033722 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:30.033648    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-d8wfv"
Jul 07 11:26:35.561727 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:35.561689    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:26:36.950630 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:36.950592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:26:38.430751 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:38.430717    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:26:45.677357 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:45.677317    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:26:52.016153 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:52.016118    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:26:52.117247 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:52.117217    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:26:59.521284 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:26:59.521248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:27:01.450892 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:27:01.450857    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:27:09.630445 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:27:09.630400    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:27:11.830945 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:27:11.830862    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:27:31.484505 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:27:31.484469    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:27:39.598445 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:27:39.598404    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:28:07.130206 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:28:07.130160    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:28:07.278329 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:28:07.278299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:28:08.306457 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:28:08.306409    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:28:08.408135 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:28:08.408102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:28:36.435479 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:28:36.435441    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:28:47.432860 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:28:47.432770    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:28:57.716080 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:28:57.716039    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:29:07.321394 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:29:07.321361    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:29:19.426881 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:29:19.426844    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:29:19.975949 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:29:19.975903    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:29:21.647983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:29:21.647944    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:29:45.126960 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:29:45.126926    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:29:56.430268 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:29:56.430226    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:30:00.145013 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:00.144982    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723730-g5qgj"]
Jul 07 11:30:00.148450 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:00.148434    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj"
Jul 07 11:30:00.151153 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:00.151133    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-fpmsd\""
Jul 07 11:30:00.155753 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:00.155718    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723730-g5qgj"]
Jul 07 11:30:00.225422 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:00.225394    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2vdf\" (UniqueName: \"kubernetes.io/projected/2bc5f507-00fe-4de9-8c24-f59641cdf494-kube-api-access-x2vdf\") pod \"maas-api-key-cleanup-29723730-g5qgj\" (UID: \"2bc5f507-00fe-4de9-8c24-f59641cdf494\") " pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj"
Jul 07 11:30:00.326429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:00.326399    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vdf\" (UniqueName: \"kubernetes.io/projected/2bc5f507-00fe-4de9-8c24-f59641cdf494-kube-api-access-x2vdf\") pod \"maas-api-key-cleanup-29723730-g5qgj\" (UID: \"2bc5f507-00fe-4de9-8c24-f59641cdf494\") " pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj"
Jul 07 11:30:00.335073 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:00.335051    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2vdf\" (UniqueName: \"kubernetes.io/projected/2bc5f507-00fe-4de9-8c24-f59641cdf494-kube-api-access-x2vdf\") pod \"maas-api-key-cleanup-29723730-g5qgj\" (UID: \"2bc5f507-00fe-4de9-8c24-f59641cdf494\") " pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj"
Jul 07 11:30:00.460555 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:00.460491    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj"
Jul 07 11:30:00.588935 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:00.588911    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723730-g5qgj"]
Jul 07 11:30:00.590942 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:30:00.590915    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc5f507_00fe_4de9_8c24_f59641cdf494.slice/crio-0e0cf941830691c927f03c9df114ea6152499236bef8634faad395e178799645 WatchSource:0}: Error finding container 0e0cf941830691c927f03c9df114ea6152499236bef8634faad395e178799645: Status 404 returned error can't find the container with id 0e0cf941830691c927f03c9df114ea6152499236bef8634faad395e178799645
Jul 07 11:30:00.947244 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:00.947187    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj" event={"ID":"2bc5f507-00fe-4de9-8c24-f59641cdf494","Type":"ContainerStarted","Data":"0e0cf941830691c927f03c9df114ea6152499236bef8634faad395e178799645"}
Jul 07 11:30:02.415975 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:02.415942    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:30:02.528444 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:02.528412    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:30:02.649991 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:02.649962    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:30:05.977055 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:05.977018    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj" event={"ID":"2bc5f507-00fe-4de9-8c24-f59641cdf494","Type":"ContainerStarted","Data":"52ef742e3c200ecae9360f7b9ff081a257948660b6c6423b370a10e938480f73"}
Jul 07 11:30:05.995138 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:05.995091    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj" podStartSLOduration=1.360420696 podStartE2EDuration="5.995076719s" totalImagesPullingTime="4.634656023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:30:05.993293329 +0000 UTC m=+887.531984743" watchObservedRunningTime="2026-07-07 11:30:05.995076719 +0000 UTC m=+887.533768130"
Jul 07 11:30:06.984323 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:06.984290    2582 generic.go:356] "Generic (PLEG): container finished" podID="2bc5f507-00fe-4de9-8c24-f59641cdf494" containerID="52ef742e3c200ecae9360f7b9ff081a257948660b6c6423b370a10e938480f73" exitCode=0
Jul 07 11:30:06.984716 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:06.984360    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj" event={"ID":"2bc5f507-00fe-4de9-8c24-f59641cdf494","Type":"ContainerDied","Data":"52ef742e3c200ecae9360f7b9ff081a257948660b6c6423b370a10e938480f73"}
Jul 07 11:30:08.124656 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:08.124629    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj"
Jul 07 11:30:08.192151 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:08.192120    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2vdf\" (UniqueName: \"kubernetes.io/projected/2bc5f507-00fe-4de9-8c24-f59641cdf494-kube-api-access-x2vdf\") pod \"2bc5f507-00fe-4de9-8c24-f59641cdf494\" (UID: \"2bc5f507-00fe-4de9-8c24-f59641cdf494\") "
Jul 07 11:30:08.194328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:08.194304    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bc5f507-00fe-4de9-8c24-f59641cdf494-kube-api-access-x2vdf" (OuterVolumeSpecName: "kube-api-access-x2vdf") pod "2bc5f507-00fe-4de9-8c24-f59641cdf494" (UID: "2bc5f507-00fe-4de9-8c24-f59641cdf494"). InnerVolumeSpecName "kube-api-access-x2vdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:30:08.293501 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:08.293426    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2vdf\" (UniqueName: \"kubernetes.io/projected/2bc5f507-00fe-4de9-8c24-f59641cdf494-kube-api-access-x2vdf\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:30:08.994002 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:08.993976    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj"
Jul 07 11:30:08.994181 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:08.993976    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723730-g5qgj" event={"ID":"2bc5f507-00fe-4de9-8c24-f59641cdf494","Type":"ContainerDied","Data":"0e0cf941830691c927f03c9df114ea6152499236bef8634faad395e178799645"}
Jul 07 11:30:08.994181 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:08.994101    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e0cf941830691c927f03c9df114ea6152499236bef8634faad395e178799645"
Jul 07 11:30:09.039746 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:09.039710    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:30:09.169369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:09.169332    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:30:09.266045 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:09.265957    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:30:09.391714 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:09.391680    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:30:15.230327 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:15.230247    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:30:15.296738 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:15.296697    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:30:18.946158 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:18.946130    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:30:18.946685 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:18.946526    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:30:31.116411 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:31.116376    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:30:39.729512 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:39.729478    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:30:49.826252 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:49.826217    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:30:57.929232 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:30:57.929186    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:31:08.634609 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:31:08.634576    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:31:17.318265 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:31:17.318231    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:31:29.628957 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:31:29.628921    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:31:39.219908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:31:39.219873    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:32:01.622235 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:32:01.622115    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:32:13.530133 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:32:13.530096    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:32:36.645225 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:32:36.645174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:32:48.523776 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:32:48.523739    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:33:03.046439 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:33:03.046404    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:33:05.227338 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:33:05.227309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:33:14.132922 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:33:14.132840    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:33:23.251406 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:33:23.251375    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:33:32.541652 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:33:32.541617    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:33:40.682189 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:33:40.682155    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:33:52.271454 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:33:52.271420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:34:08.323964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:08.323920    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"]
Jul 07 11:34:08.324465 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:08.324130    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x" podUID="fdd8cab2-fdf7-4b35-a793-62cc853623c5" containerName="manager" containerID="cri-o://0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35" gracePeriod=10
Jul 07 11:34:10.980882 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:10.980860    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:34:11.070543 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.070508    2582 generic.go:356] "Generic (PLEG): container finished" podID="fdd8cab2-fdf7-4b35-a793-62cc853623c5" containerID="0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35" exitCode=0
Jul 07 11:34:11.070694 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.070556    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"
Jul 07 11:34:11.070694 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.070568    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x" event={"ID":"fdd8cab2-fdf7-4b35-a793-62cc853623c5","Type":"ContainerDied","Data":"0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35"}
Jul 07 11:34:11.070694 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.070621    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x" event={"ID":"fdd8cab2-fdf7-4b35-a793-62cc853623c5","Type":"ContainerDied","Data":"878bdd913bc2a2f6e1ac22eed05038e2c382dac729d48d7c1ddf0bf9fb8f833e"}
Jul 07 11:34:11.070694 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.070651    2582 scope.go:117] "RemoveContainer" containerID="0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35"
Jul 07 11:34:11.079710 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.079691    2582 scope.go:117] "RemoveContainer" containerID="0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35"
Jul 07 11:34:11.079950 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:34:11.079931    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35\": container with ID starting with 0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35 not found: ID does not exist" containerID="0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35"
Jul 07 11:34:11.080004 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.079959    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35"} err="failed to get container status \"0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35\": rpc error: code = NotFound desc = could not find container \"0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35\": container with ID starting with 0cf4787238bed373b2111b7c3142744c9d3dcc1dd9b09c179f8116c3477b7c35 not found: ID does not exist"
Jul 07 11:34:11.106332 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.106282    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdd8cab2-fdf7-4b35-a793-62cc853623c5-extensions-socket-volume\") pod \"fdd8cab2-fdf7-4b35-a793-62cc853623c5\" (UID: \"fdd8cab2-fdf7-4b35-a793-62cc853623c5\") "
Jul 07 11:34:11.106436 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.106420    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9s6q\" (UniqueName: \"kubernetes.io/projected/fdd8cab2-fdf7-4b35-a793-62cc853623c5-kube-api-access-t9s6q\") pod \"fdd8cab2-fdf7-4b35-a793-62cc853623c5\" (UID: \"fdd8cab2-fdf7-4b35-a793-62cc853623c5\") "
Jul 07 11:34:11.106682 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.106661    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdd8cab2-fdf7-4b35-a793-62cc853623c5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fdd8cab2-fdf7-4b35-a793-62cc853623c5" (UID: "fdd8cab2-fdf7-4b35-a793-62cc853623c5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:34:11.108406 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.108374    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdd8cab2-fdf7-4b35-a793-62cc853623c5-kube-api-access-t9s6q" (OuterVolumeSpecName: "kube-api-access-t9s6q") pod "fdd8cab2-fdf7-4b35-a793-62cc853623c5" (UID: "fdd8cab2-fdf7-4b35-a793-62cc853623c5"). InnerVolumeSpecName "kube-api-access-t9s6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:34:11.207505 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.207478    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9s6q\" (UniqueName: \"kubernetes.io/projected/fdd8cab2-fdf7-4b35-a793-62cc853623c5-kube-api-access-t9s6q\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:34:11.207505 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.207501    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdd8cab2-fdf7-4b35-a793-62cc853623c5-extensions-socket-volume\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:34:11.395691 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.395663    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"]
Jul 07 11:34:11.398117 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:11.398093    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rp29x"]
Jul 07 11:34:13.025074 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:13.025038    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdd8cab2-fdf7-4b35-a793-62cc853623c5" path="/var/lib/kubelet/pods/fdd8cab2-fdf7-4b35-a793-62cc853623c5/volumes"
Jul 07 11:34:38.197110 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:34:38.197072    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:35:09.509799 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.509712    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"]
Jul 07 11:35:09.510397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.510185    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc5f507-00fe-4de9-8c24-f59641cdf494" containerName="cleanup"
Jul 07 11:35:09.510397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.510222    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc5f507-00fe-4de9-8c24-f59641cdf494" containerName="cleanup"
Jul 07 11:35:09.510397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.510243    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdd8cab2-fdf7-4b35-a793-62cc853623c5" containerName="manager"
Jul 07 11:35:09.510397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.510251    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdd8cab2-fdf7-4b35-a793-62cc853623c5" containerName="manager"
Jul 07 11:35:09.510397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.510359    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdd8cab2-fdf7-4b35-a793-62cc853623c5" containerName="manager"
Jul 07 11:35:09.510397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.510372    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bc5f507-00fe-4de9-8c24-f59641cdf494" containerName="cleanup"
Jul 07 11:35:09.513441 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.513420    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"
Jul 07 11:35:09.518802 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.518782    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d8l4g\""
Jul 07 11:35:09.544326 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.544303    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"]
Jul 07 11:35:09.670678 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.670645    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31665f10-91e9-4f6a-a2f6-1f87e8389d48-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7rqcr\" (UID: \"31665f10-91e9-4f6a-a2f6-1f87e8389d48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"
Jul 07 11:35:09.670842 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.670717    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zxrz\" (UniqueName: \"kubernetes.io/projected/31665f10-91e9-4f6a-a2f6-1f87e8389d48-kube-api-access-5zxrz\") pod \"kuadrant-operator-controller-manager-86878c769f-7rqcr\" (UID: \"31665f10-91e9-4f6a-a2f6-1f87e8389d48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"
Jul 07 11:35:09.771993 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.771912    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31665f10-91e9-4f6a-a2f6-1f87e8389d48-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7rqcr\" (UID: \"31665f10-91e9-4f6a-a2f6-1f87e8389d48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"
Jul 07 11:35:09.771993 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.771968    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zxrz\" (UniqueName: \"kubernetes.io/projected/31665f10-91e9-4f6a-a2f6-1f87e8389d48-kube-api-access-5zxrz\") pod \"kuadrant-operator-controller-manager-86878c769f-7rqcr\" (UID: \"31665f10-91e9-4f6a-a2f6-1f87e8389d48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"
Jul 07 11:35:09.772371 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.772349    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31665f10-91e9-4f6a-a2f6-1f87e8389d48-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7rqcr\" (UID: \"31665f10-91e9-4f6a-a2f6-1f87e8389d48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"
Jul 07 11:35:09.782068 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.782043    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zxrz\" (UniqueName: \"kubernetes.io/projected/31665f10-91e9-4f6a-a2f6-1f87e8389d48-kube-api-access-5zxrz\") pod \"kuadrant-operator-controller-manager-86878c769f-7rqcr\" (UID: \"31665f10-91e9-4f6a-a2f6-1f87e8389d48\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"
Jul 07 11:35:09.823429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.823405    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"
Jul 07 11:35:09.968338 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:09.968308    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"]
Jul 07 11:35:09.970047 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:35:09.970014    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31665f10_91e9_4f6a_a2f6_1f87e8389d48.slice/crio-406005fa786c4dd289bca25d4783387b4a4c005f0db3e7d5923ad8c29420e7e8 WatchSource:0}: Error finding container 406005fa786c4dd289bca25d4783387b4a4c005f0db3e7d5923ad8c29420e7e8: Status 404 returned error can't find the container with id 406005fa786c4dd289bca25d4783387b4a4c005f0db3e7d5923ad8c29420e7e8
Jul 07 11:35:10.325339 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:10.325252    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr" event={"ID":"31665f10-91e9-4f6a-a2f6-1f87e8389d48","Type":"ContainerStarted","Data":"2654ca4895551052c7ee3d943ab6301df1dd555c4997addb4f578302825fb3f0"}
Jul 07 11:35:10.325339 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:10.325304    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr" event={"ID":"31665f10-91e9-4f6a-a2f6-1f87e8389d48","Type":"ContainerStarted","Data":"406005fa786c4dd289bca25d4783387b4a4c005f0db3e7d5923ad8c29420e7e8"}
Jul 07 11:35:10.325339 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:10.325335    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"
Jul 07 11:35:10.372019 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:10.371972    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr" podStartSLOduration=1.371957512 podStartE2EDuration="1.371957512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:35:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:35:10.370543333 +0000 UTC m=+1191.909234746" watchObservedRunningTime="2026-07-07 11:35:10.371957512 +0000 UTC m=+1191.910648986"
Jul 07 11:35:18.972183 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:18.972156    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:35:18.974145 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:18.974124    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:35:21.333056 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:21.333031    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7rqcr"
Jul 07 11:35:29.240867 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:29.240833    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:35:40.418015 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:40.417985    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:35:51.629328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:51.629291    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:35:59.231088 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:35:59.231057    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:36:10.429893 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:36:10.429860    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:36:19.025454 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:36:19.025414    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:36:29.820445 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:36:29.820409    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:36:39.828729 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:36:39.828695    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:36:49.739803 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:36:49.739766    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:37:00.018827 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:37:00.018790    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:37:12.421704 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:37:12.421665    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:37:19.726790 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:37:19.726749    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:37:40.519359 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:37:40.519325    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:37:47.325300 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:37:47.325213    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:37:56.221608 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:37:56.221574    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:38:08.217946 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:38:08.217912    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:38:23.676126 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:38:23.676092    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:38:34.624584 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:38:34.624553    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:38:44.329745 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:38:44.329706    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:38:53.925099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:38:53.925062    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:39:03.374952 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:39:03.374915    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:39:12.131018 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:39:12.130935    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:39:24.024014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:39:24.023984    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:39:31.319639 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:39:31.319602    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:39:39.930908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:39:39.930873    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:39:48.725180 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:39:48.725145    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:40:02.118639 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:40:02.118595    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:40:11.727416 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:40:11.727382    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:40:19.001212 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:40:19.001172    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:40:19.003897 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:40:19.003868    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:40:20.242781 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:40:20.242746    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:40:33.018540 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:40:33.018508    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:40:44.966306 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:40:44.966223    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:40:53.746402 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:40:53.746369    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:40:55.195687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:40:55.195653    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:40:57.531658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:40:57.531626    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:41:00.876973 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:00.876935    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:06.259146 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:06.259112    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:41:28.606798 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:28.606757    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:32.393889 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:32.393853    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:41:36.717121 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:36.717082    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:41.247292 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:41.247258    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:41:41.949847 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:41.949805    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:44.821639 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:44.821607    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:44.946866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:44.946829    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:45.268555 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:45.268523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:50.642312 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:50.642276    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:41:51.205290 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:51.205249    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:51.335605 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:51.335567    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:51.504058 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:51.504027    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:51.602870 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:51.602837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:52.653957 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.653928    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:52.766969 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.766928    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:52.836421 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.836392    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"]
Jul 07 11:41:52.839764 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.839741    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:52.842976 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.842955    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-d21d182b-serving-cert\""
Jul 07 11:41:52.843186 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.843169    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-fpmsd\""
Jul 07 11:41:52.872069 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.872043    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"]
Jul 07 11:41:52.893276 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.893242    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:52.927948 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.927863    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13d02a49-fb05-40f4-9882-1493dd071555-openshift-service-ca\") pod \"maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") " pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:52.927948 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.927917    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htt86\" (UniqueName: \"kubernetes.io/projected/13d02a49-fb05-40f4-9882-1493dd071555-kube-api-access-htt86\") pod \"maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") " pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:52.927948 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:52.927946    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d02a49-fb05-40f4-9882-1493dd071555-maas-api-tls\") pod \"maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") " pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:53.028331 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.028299    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13d02a49-fb05-40f4-9882-1493dd071555-openshift-service-ca\") pod \"maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") " pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:53.028527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.028346    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htt86\" (UniqueName: \"kubernetes.io/projected/13d02a49-fb05-40f4-9882-1493dd071555-kube-api-access-htt86\") pod \"maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") " pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:53.028527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.028370    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d02a49-fb05-40f4-9882-1493dd071555-maas-api-tls\") pod \"maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") " pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:53.028527 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:41:53.028501    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-d21d182b-serving-cert: secret "maas-api-e2e-ait-d21d182b-serving-cert" not found
Jul 07 11:41:53.028704 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:41:53.028564    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13d02a49-fb05-40f4-9882-1493dd071555-maas-api-tls podName:13d02a49-fb05-40f4-9882-1493dd071555 nodeName:}" failed. No retries permitted until 2026-07-07 11:41:53.528544805 +0000 UTC m=+1595.067236200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/13d02a49-fb05-40f4-9882-1493dd071555-maas-api-tls") pod "maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz" (UID: "13d02a49-fb05-40f4-9882-1493dd071555") : secret "maas-api-e2e-ait-d21d182b-serving-cert" not found
Jul 07 11:41:53.029052 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.029026    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13d02a49-fb05-40f4-9882-1493dd071555-openshift-service-ca\") pod \"maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") " pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:53.060774 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.060736    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htt86\" (UniqueName: \"kubernetes.io/projected/13d02a49-fb05-40f4-9882-1493dd071555-kube-api-access-htt86\") pod \"maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") " pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:53.163762 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.163706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:53.532029 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.531992    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d02a49-fb05-40f4-9882-1493dd071555-maas-api-tls\") pod \"maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") " pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:53.534588 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.534563    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d02a49-fb05-40f4-9882-1493dd071555-maas-api-tls\") pod \"maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") " pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:53.620423 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.620391    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:53.748627 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.748597    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:53.751449 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:53.751431    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:54.101419 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:54.101393    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"]
Jul 07 11:41:54.103675 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:41:54.103649    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d02a49_fb05_40f4_9882_1493dd071555.slice/crio-49f21cbd3b3408801c85ed60690626f5eabdf30173a1c1e8165dca9696f04bd7 WatchSource:0}: Error finding container 49f21cbd3b3408801c85ed60690626f5eabdf30173a1c1e8165dca9696f04bd7: Status 404 returned error can't find the container with id 49f21cbd3b3408801c85ed60690626f5eabdf30173a1c1e8165dca9696f04bd7
Jul 07 11:41:54.105091 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:54.105076    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:41:54.143576 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:54.143548    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz" event={"ID":"13d02a49-fb05-40f4-9882-1493dd071555","Type":"ContainerStarted","Data":"49f21cbd3b3408801c85ed60690626f5eabdf30173a1c1e8165dca9696f04bd7"}
Jul 07 11:41:55.134290 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:55.134254    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:57.163252 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:57.163208    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz" event={"ID":"13d02a49-fb05-40f4-9882-1493dd071555","Type":"ContainerStarted","Data":"e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c"}
Jul 07 11:41:57.163718 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:57.163304    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:57.189901 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:57.189857    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz" podStartSLOduration=2.866179501 podStartE2EDuration="5.189843147s" totalImagesPullingTime="2.323663646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:57.189008825 +0000 UTC m=+1598.727700258" watchObservedRunningTime="2026-07-07 11:41:57.189843147 +0000 UTC m=+1598.728534558"
Jul 07 11:41:57.537927 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:57.537897    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:57.904859 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:57.904777    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:58.120919 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:58.120889    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:58.232734 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:58.232702    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:58.351906 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:58.351874    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"]
Jul 07 11:41:58.410988 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:58.410955    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:41:59.173468 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.173428    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz" podUID="13d02a49-fb05-40f4-9882-1493dd071555" containerName="maas-api" containerID="cri-o://e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c" gracePeriod=30
Jul 07 11:41:59.424068 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.424015    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:41:59.483419 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.483397    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13d02a49-fb05-40f4-9882-1493dd071555-openshift-service-ca\") pod \"13d02a49-fb05-40f4-9882-1493dd071555\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") "
Jul 07 11:41:59.483615 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.483427    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htt86\" (UniqueName: \"kubernetes.io/projected/13d02a49-fb05-40f4-9882-1493dd071555-kube-api-access-htt86\") pod \"13d02a49-fb05-40f4-9882-1493dd071555\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") "
Jul 07 11:41:59.483615 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.483480    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d02a49-fb05-40f4-9882-1493dd071555-maas-api-tls\") pod \"13d02a49-fb05-40f4-9882-1493dd071555\" (UID: \"13d02a49-fb05-40f4-9882-1493dd071555\") "
Jul 07 11:41:59.483712 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.483688    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13d02a49-fb05-40f4-9882-1493dd071555-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13d02a49-fb05-40f4-9882-1493dd071555" (UID: "13d02a49-fb05-40f4-9882-1493dd071555"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:41:59.485553 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.485524    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13d02a49-fb05-40f4-9882-1493dd071555-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13d02a49-fb05-40f4-9882-1493dd071555" (UID: "13d02a49-fb05-40f4-9882-1493dd071555"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:41:59.485655 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.485546    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d02a49-fb05-40f4-9882-1493dd071555-kube-api-access-htt86" (OuterVolumeSpecName: "kube-api-access-htt86") pod "13d02a49-fb05-40f4-9882-1493dd071555" (UID: "13d02a49-fb05-40f4-9882-1493dd071555"). InnerVolumeSpecName "kube-api-access-htt86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:41:59.584049 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.584024    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13d02a49-fb05-40f4-9882-1493dd071555-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:41:59.584049 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.584050    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13d02a49-fb05-40f4-9882-1493dd071555-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:41:59.584268 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:41:59.584063    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htt86\" (UniqueName: \"kubernetes.io/projected/13d02a49-fb05-40f4-9882-1493dd071555-kube-api-access-htt86\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:00.178806 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:00.178768    2582 generic.go:356] "Generic (PLEG): container finished" podID="13d02a49-fb05-40f4-9882-1493dd071555" containerID="e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c" exitCode=0
Jul 07 11:42:00.178968 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:00.178823    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz" event={"ID":"13d02a49-fb05-40f4-9882-1493dd071555","Type":"ContainerDied","Data":"e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c"}
Jul 07 11:42:00.178968 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:00.178831    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"
Jul 07 11:42:00.178968 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:00.178872    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz" event={"ID":"13d02a49-fb05-40f4-9882-1493dd071555","Type":"ContainerDied","Data":"49f21cbd3b3408801c85ed60690626f5eabdf30173a1c1e8165dca9696f04bd7"}
Jul 07 11:42:00.178968 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:00.178899    2582 scope.go:117] "RemoveContainer" containerID="e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c"
Jul 07 11:42:00.187685 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:00.187668    2582 scope.go:117] "RemoveContainer" containerID="e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c"
Jul 07 11:42:00.187907 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:42:00.187890    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c\": container with ID starting with e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c not found: ID does not exist" containerID="e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c"
Jul 07 11:42:00.187950 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:00.187914    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c"} err="failed to get container status \"e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c\": rpc error: code = NotFound desc = could not find container \"e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c\": container with ID starting with e6efcf7a12ca72a2b326f6acd901447859298a0de00fab15885700065e87957c not found: ID does not exist"
Jul 07 11:42:00.204299 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:00.204274    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"]
Jul 07 11:42:00.208534 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:00.208512    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-d21d182b-5f8c944694-2jwdz"]
Jul 07 11:42:01.025825 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:01.025792    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13d02a49-fb05-40f4-9882-1493dd071555" path="/var/lib/kubelet/pods/13d02a49-fb05-40f4-9882-1493dd071555/volumes"
Jul 07 11:42:01.769139 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:01.769095    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:04.782106 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:04.782067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:04.885582 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:04.885549    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:05.006405 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:05.006363    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:05.106954 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:05.106872    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:06.305395 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.305357    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:06.450969 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.450936    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"]
Jul 07 11:42:06.451451 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.451432    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13d02a49-fb05-40f4-9882-1493dd071555" containerName="maas-api"
Jul 07 11:42:06.451451 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.451451    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="13d02a49-fb05-40f4-9882-1493dd071555" containerName="maas-api"
Jul 07 11:42:06.451618 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.451582    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="13d02a49-fb05-40f4-9882-1493dd071555" containerName="maas-api"
Jul 07 11:42:06.459162 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.459140    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:06.462267 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.462246    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-fpmsd\""
Jul 07 11:42:06.462494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.462479    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-2813d1c1-serving-cert\""
Jul 07 11:42:06.472735 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.472707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:06.474687 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.474665    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"]
Jul 07 11:42:06.541208 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.541173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a22f7025-819c-4ad9-a102-cefbe81a6a36-maas-api-tls\") pod \"maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") " pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:06.541319 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.541240    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g8l4\" (UniqueName: \"kubernetes.io/projected/a22f7025-819c-4ad9-a102-cefbe81a6a36-kube-api-access-9g8l4\") pod \"maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") " pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:06.541319 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.541264    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a22f7025-819c-4ad9-a102-cefbe81a6a36-openshift-service-ca\") pod \"maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") " pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:06.641696 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.641631    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g8l4\" (UniqueName: \"kubernetes.io/projected/a22f7025-819c-4ad9-a102-cefbe81a6a36-kube-api-access-9g8l4\") pod \"maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") " pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:06.641696 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.641674    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a22f7025-819c-4ad9-a102-cefbe81a6a36-openshift-service-ca\") pod \"maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") " pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:06.641870 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.641733    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a22f7025-819c-4ad9-a102-cefbe81a6a36-maas-api-tls\") pod \"maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") " pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:06.641870 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:42:06.641849    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-2813d1c1-serving-cert: secret "maas-api-e2e-ait-2813d1c1-serving-cert" not found
Jul 07 11:42:06.641993 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:42:06.641911    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a22f7025-819c-4ad9-a102-cefbe81a6a36-maas-api-tls podName:a22f7025-819c-4ad9-a102-cefbe81a6a36 nodeName:}" failed. No retries permitted until 2026-07-07 11:42:07.1418924 +0000 UTC m=+1608.680583796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a22f7025-819c-4ad9-a102-cefbe81a6a36-maas-api-tls") pod "maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb" (UID: "a22f7025-819c-4ad9-a102-cefbe81a6a36") : secret "maas-api-e2e-ait-2813d1c1-serving-cert" not found
Jul 07 11:42:06.642417 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.642396    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a22f7025-819c-4ad9-a102-cefbe81a6a36-openshift-service-ca\") pod \"maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") " pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:06.661574 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.661550    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g8l4\" (UniqueName: \"kubernetes.io/projected/a22f7025-819c-4ad9-a102-cefbe81a6a36-kube-api-access-9g8l4\") pod \"maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") " pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:06.681052 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.681029    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:06.805477 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:06.805444    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:07.146306 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:07.146273    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a22f7025-819c-4ad9-a102-cefbe81a6a36-maas-api-tls\") pod \"maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") " pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:07.148680 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:07.148660    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a22f7025-819c-4ad9-a102-cefbe81a6a36-maas-api-tls\") pod \"maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") " pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:07.369728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:07.369681    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:07.372403 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:07.372381    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:07.470953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:07.470917    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:07.515665 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:07.515631    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"]
Jul 07 11:42:07.516429 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:42:07.516404    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda22f7025_819c_4ad9_a102_cefbe81a6a36.slice/crio-418ad8ea2caa5c0d0a991f45ff4709c160b502791aa765bb9059cc2081017923 WatchSource:0}: Error finding container 418ad8ea2caa5c0d0a991f45ff4709c160b502791aa765bb9059cc2081017923: Status 404 returned error can't find the container with id 418ad8ea2caa5c0d0a991f45ff4709c160b502791aa765bb9059cc2081017923
Jul 07 11:42:08.139603 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:08.139574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:08.215635 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:08.215606    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb" event={"ID":"a22f7025-819c-4ad9-a102-cefbe81a6a36","Type":"ContainerStarted","Data":"1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7"}
Jul 07 11:42:08.215766 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:08.215656    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb" event={"ID":"a22f7025-819c-4ad9-a102-cefbe81a6a36","Type":"ContainerStarted","Data":"418ad8ea2caa5c0d0a991f45ff4709c160b502791aa765bb9059cc2081017923"}
Jul 07 11:42:08.215766 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:08.215712    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:08.243664 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:08.243618    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb" podStartSLOduration=1.928594465 podStartE2EDuration="2.243603747s" totalImagesPullingTime="315.009282ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:42:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:42:08.24011623 +0000 UTC m=+1609.778807643" watchObservedRunningTime="2026-07-07 11:42:08.243603747 +0000 UTC m=+1609.782295198"
Jul 07 11:42:11.318308 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.318216    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:11.672960 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.672881    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"]
Jul 07 11:42:11.673097 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.673074    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb" podUID="a22f7025-819c-4ad9-a102-cefbe81a6a36" containerName="maas-api" containerID="cri-o://1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7" gracePeriod=30
Jul 07 11:42:11.714878 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.714850    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:11.918572 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.918547    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:11.986222 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.986178    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a22f7025-819c-4ad9-a102-cefbe81a6a36-maas-api-tls\") pod \"a22f7025-819c-4ad9-a102-cefbe81a6a36\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") "
Jul 07 11:42:11.986370 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.986239    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9g8l4\" (UniqueName: \"kubernetes.io/projected/a22f7025-819c-4ad9-a102-cefbe81a6a36-kube-api-access-9g8l4\") pod \"a22f7025-819c-4ad9-a102-cefbe81a6a36\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") "
Jul 07 11:42:11.986370 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.986264    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a22f7025-819c-4ad9-a102-cefbe81a6a36-openshift-service-ca\") pod \"a22f7025-819c-4ad9-a102-cefbe81a6a36\" (UID: \"a22f7025-819c-4ad9-a102-cefbe81a6a36\") "
Jul 07 11:42:11.986666 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.986638    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a22f7025-819c-4ad9-a102-cefbe81a6a36-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a22f7025-819c-4ad9-a102-cefbe81a6a36" (UID: "a22f7025-819c-4ad9-a102-cefbe81a6a36"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:42:11.988238 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.988207    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a22f7025-819c-4ad9-a102-cefbe81a6a36-kube-api-access-9g8l4" (OuterVolumeSpecName: "kube-api-access-9g8l4") pod "a22f7025-819c-4ad9-a102-cefbe81a6a36" (UID: "a22f7025-819c-4ad9-a102-cefbe81a6a36"). InnerVolumeSpecName "kube-api-access-9g8l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:42:11.988346 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:11.988300    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a22f7025-819c-4ad9-a102-cefbe81a6a36-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a22f7025-819c-4ad9-a102-cefbe81a6a36" (UID: "a22f7025-819c-4ad9-a102-cefbe81a6a36"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:42:12.087607 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.087572    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a22f7025-819c-4ad9-a102-cefbe81a6a36-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:12.087607 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.087602    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g8l4\" (UniqueName: \"kubernetes.io/projected/a22f7025-819c-4ad9-a102-cefbe81a6a36-kube-api-access-9g8l4\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:12.087804 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.087617    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a22f7025-819c-4ad9-a102-cefbe81a6a36-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:12.234331 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.234296    2582 generic.go:356] "Generic (PLEG): container finished" podID="a22f7025-819c-4ad9-a102-cefbe81a6a36" containerID="1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7" exitCode=0
Jul 07 11:42:12.234488 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.234338    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"
Jul 07 11:42:12.234488 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.234382    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb" event={"ID":"a22f7025-819c-4ad9-a102-cefbe81a6a36","Type":"ContainerDied","Data":"1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7"}
Jul 07 11:42:12.234488 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.234435    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb" event={"ID":"a22f7025-819c-4ad9-a102-cefbe81a6a36","Type":"ContainerDied","Data":"418ad8ea2caa5c0d0a991f45ff4709c160b502791aa765bb9059cc2081017923"}
Jul 07 11:42:12.234488 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.234458    2582 scope.go:117] "RemoveContainer" containerID="1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7"
Jul 07 11:42:12.243943 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.243925    2582 scope.go:117] "RemoveContainer" containerID="1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7"
Jul 07 11:42:12.244218 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:42:12.244181    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7\": container with ID starting with 1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7 not found: ID does not exist" containerID="1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7"
Jul 07 11:42:12.244328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.244241    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7"} err="failed to get container status \"1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7\": rpc error: code = NotFound desc = could not find container \"1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7\": container with ID starting with 1613273dce999d490402f48c60fbe33627f045bdc4115da3fa1c2857e39babb7 not found: ID does not exist"
Jul 07 11:42:12.271181 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.271154    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"]
Jul 07 11:42:12.278499 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:12.278479    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-2813d1c1-b4766789f-m2wsb"]
Jul 07 11:42:13.025588 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:13.025555    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a22f7025-819c-4ad9-a102-cefbe81a6a36" path="/var/lib/kubelet/pods/a22f7025-819c-4ad9-a102-cefbe81a6a36/volumes"
Jul 07 11:42:15.097490 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:15.097452    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:17.715270 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:17.715239    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:17.836084 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:17.836052    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:24.114745 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.114707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:24.218632 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.218597    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:24.434147 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.434119    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"]
Jul 07 11:42:24.434551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.434525    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a22f7025-819c-4ad9-a102-cefbe81a6a36" containerName="maas-api"
Jul 07 11:42:24.434551 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.434545    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a22f7025-819c-4ad9-a102-cefbe81a6a36" containerName="maas-api"
Jul 07 11:42:24.434714 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.434615    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a22f7025-819c-4ad9-a102-cefbe81a6a36" containerName="maas-api"
Jul 07 11:42:24.437617 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.437599    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.442929 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.442906    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 11:42:24.443082 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.443057    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-bdcf9e68-openshift-default-dockercfg-ndrmk\""
Jul 07 11:42:24.546011 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.545976    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:24.592831 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.592805    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-envoy\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.592965 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.592845    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-podinfo\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.592965 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.592894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ee6b677-5b10-4179-b799-d3d984c25a4d-istiod-ca-cert\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.592965 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.592926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-credential-socket\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.592965 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.592962    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-socket\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.593140 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.593002    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkdcs\" (UniqueName: \"kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-kube-api-access-qkdcs\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.593140 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.593024    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-certs\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.593140 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.593073    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-token\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.593140 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.593132    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-data\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.598441 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.598418    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"]
Jul 07 11:42:24.694411 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694352    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-token\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.694411 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694385    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-data\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.694576 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-envoy\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.694576 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-podinfo\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.694576 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ee6b677-5b10-4179-b799-d3d984c25a4d-istiod-ca-cert\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.694576 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694524    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-credential-socket\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.694576 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694556    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-socket\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.694838 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694594    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkdcs\" (UniqueName: \"kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-kube-api-access-qkdcs\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.694838 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694625    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-certs\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.694838 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694812    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-data\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.695135 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.694962    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-socket\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.695135 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.695042    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-certs\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.695256 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.695188    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-credential-socket\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.695308 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.695290    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ee6b677-5b10-4179-b799-d3d984c25a4d-istiod-ca-cert\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.696828 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.696806    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-envoy\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.696920 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.696891    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-podinfo\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.713799 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.713778    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkdcs\" (UniqueName: \"kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-kube-api-access-qkdcs\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.718853 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.718833    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-token\") pod \"e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") " pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.747926 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.747904    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:24.909666 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:24.909640    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"]
Jul 07 11:42:24.910510 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:42:24.910487    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee6b677_5b10_4179_b799_d3d984c25a4d.slice/crio-17770470a10bfb6e39fcc9074b25aa899cdbad94d4df10a54f76bbc44eac06a2 WatchSource:0}: Error finding container 17770470a10bfb6e39fcc9074b25aa899cdbad94d4df10a54f76bbc44eac06a2: Status 404 returned error can't find the container with id 17770470a10bfb6e39fcc9074b25aa899cdbad94d4df10a54f76bbc44eac06a2
Jul 07 11:42:25.218891 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.218849    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:25.292165 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.292135    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh" event={"ID":"8ee6b677-5b10-4179-b799-d3d984c25a4d","Type":"ContainerStarted","Data":"17770470a10bfb6e39fcc9074b25aa899cdbad94d4df10a54f76bbc44eac06a2"}
Jul 07 11:42:25.337929 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.337854    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:25.702292 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.702221    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:25.800478 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.800382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:25.923077 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.922777    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"]
Jul 07 11:42:25.926681 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.926655    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:25.929993 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.929935    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-bdcf9e68-serving-cert\""
Jul 07 11:42:25.930135 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.929974    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-fpmsd\""
Jul 07 11:42:25.940937 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.940876    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"]
Jul 07 11:42:25.980664 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:25.980639    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:26.006494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.006469    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-openshift-service-ca\") pod \"maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") " pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:26.006633 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.006530    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rntn4\" (UniqueName: \"kubernetes.io/projected/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-kube-api-access-rntn4\") pod \"maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") " pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:26.006720 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.006697    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-maas-api-tls\") pod \"maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") " pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:26.107158 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.107126    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-openshift-service-ca\") pod \"maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") " pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:26.107344 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.107220    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rntn4\" (UniqueName: \"kubernetes.io/projected/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-kube-api-access-rntn4\") pod \"maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") " pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:26.107344 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.107300    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-maas-api-tls\") pod \"maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") " pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:26.107477 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:42:26.107414    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-bdcf9e68-serving-cert: secret "maas-api-e2e-derive-bdcf9e68-serving-cert" not found
Jul 07 11:42:26.107477 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:42:26.107474    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-maas-api-tls podName:7d8a3b4b-abb4-4dd0-ac55-407eaea08897 nodeName:}" failed. No retries permitted until 2026-07-07 11:42:26.607457806 +0000 UTC m=+1628.146149219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-maas-api-tls") pod "maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8" (UID: "7d8a3b4b-abb4-4dd0-ac55-407eaea08897") : secret "maas-api-e2e-derive-bdcf9e68-serving-cert" not found
Jul 07 11:42:26.107950 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.107924    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-openshift-service-ca\") pod \"maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") " pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:26.122180 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.122159    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rntn4\" (UniqueName: \"kubernetes.io/projected/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-kube-api-access-rntn4\") pod \"maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") " pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:26.335938 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.335836    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:26.528568 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.528525    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:26.612932 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.612852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-maas-api-tls\") pod \"maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") " pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:26.615664 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.615633    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-maas-api-tls\") pod \"maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") " pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:26.632336 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.632308    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:26.841948 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:26.841913    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:27.514575 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:27.514532    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:42:27.514867 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:27.514612    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:42:27.514867 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:27.514650    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:42:27.571594 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:27.571551    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"]
Jul 07 11:42:27.572531 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:42:27.572497    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d8a3b4b_abb4_4dd0_ac55_407eaea08897.slice/crio-11850440921fbb3291dcb08e7d9e629621f78aa1ca1ed3fd7455936fd0a7f323 WatchSource:0}: Error finding container 11850440921fbb3291dcb08e7d9e629621f78aa1ca1ed3fd7455936fd0a7f323: Status 404 returned error can't find the container with id 11850440921fbb3291dcb08e7d9e629621f78aa1ca1ed3fd7455936fd0a7f323
Jul 07 11:42:28.309166 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:28.309131    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8" event={"ID":"7d8a3b4b-abb4-4dd0-ac55-407eaea08897","Type":"ContainerStarted","Data":"c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a"}
Jul 07 11:42:28.309367 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:28.309180    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8" event={"ID":"7d8a3b4b-abb4-4dd0-ac55-407eaea08897","Type":"ContainerStarted","Data":"11850440921fbb3291dcb08e7d9e629621f78aa1ca1ed3fd7455936fd0a7f323"}
Jul 07 11:42:28.309367 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:28.309239    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:28.310842 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:28.310819    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh" event={"ID":"8ee6b677-5b10-4179-b799-d3d984c25a4d","Type":"ContainerStarted","Data":"122a49ff4725d4cf34264f71645b1660b969c8b609acc2e422536152818fc361"}
Jul 07 11:42:28.327678 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:28.327636    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8" podStartSLOduration=2.966104226 podStartE2EDuration="3.327622516s" totalImagesPullingTime="361.51829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:42:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:42:28.32617118 +0000 UTC m=+1629.864862589" watchObservedRunningTime="2026-07-07 11:42:28.327622516 +0000 UTC m=+1629.866313929"
Jul 07 11:42:28.349030 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:28.348951    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh" podStartSLOduration=1.747134838 podStartE2EDuration="4.348933734s" totalImagesPullingTime="2.601798896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:42:28.348246931 +0000 UTC m=+1629.886938337" watchObservedRunningTime="2026-07-07 11:42:28.348933734 +0000 UTC m=+1629.887625147"
Jul 07 11:42:28.635050 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:28.634978    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:42:28.698032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:28.698003    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:28.748601 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:28.748553    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:28.753746 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:28.753716    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:29.316067 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:29.316039    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:29.316981 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:29.316964    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:30.095297 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.095253    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:30.232297 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.232267    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"]
Jul 07 11:42:30.268001 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.267960    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:30.320321 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.320289    2582 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh" secret="" err="secret \"e2e-derive-bdcf9e68-openshift-default-dockercfg-ndrmk\" not found"
Jul 07 11:42:30.363655 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.363571    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:30.396502 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.396472    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"]
Jul 07 11:42:30.396684 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.396653    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8" podUID="7d8a3b4b-abb4-4dd0-ac55-407eaea08897" containerName="maas-api" containerID="cri-o://c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a" gracePeriod=30
Jul 07 11:42:30.477219 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.473738    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:30.700346 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.700317    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:30.852835 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.852800    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-openshift-service-ca\") pod \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") "
Jul 07 11:42:30.853011 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.852847    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rntn4\" (UniqueName: \"kubernetes.io/projected/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-kube-api-access-rntn4\") pod \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") "
Jul 07 11:42:30.853011 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.852935    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-maas-api-tls\") pod \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\" (UID: \"7d8a3b4b-abb4-4dd0-ac55-407eaea08897\") "
Jul 07 11:42:30.853254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.853219    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d8a3b4b-abb4-4dd0-ac55-407eaea08897" (UID: "7d8a3b4b-abb4-4dd0-ac55-407eaea08897"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:42:30.855041 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.855022    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-kube-api-access-rntn4" (OuterVolumeSpecName: "kube-api-access-rntn4") pod "7d8a3b4b-abb4-4dd0-ac55-407eaea08897" (UID: "7d8a3b4b-abb4-4dd0-ac55-407eaea08897"). InnerVolumeSpecName "kube-api-access-rntn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:42:30.855112 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.855019    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d8a3b4b-abb4-4dd0-ac55-407eaea08897" (UID: "7d8a3b4b-abb4-4dd0-ac55-407eaea08897"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:42:30.954622 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.954535    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:30.954622 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.954562    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:30.954622 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:30.954574    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rntn4\" (UniqueName: \"kubernetes.io/projected/7d8a3b4b-abb4-4dd0-ac55-407eaea08897-kube-api-access-rntn4\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:31.325476 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:31.325441    2582 generic.go:356] "Generic (PLEG): container finished" podID="7d8a3b4b-abb4-4dd0-ac55-407eaea08897" containerID="c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a" exitCode=0
Jul 07 11:42:31.325955 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:31.325492    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"
Jul 07 11:42:31.325955 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:31.325536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8" event={"ID":"7d8a3b4b-abb4-4dd0-ac55-407eaea08897","Type":"ContainerDied","Data":"c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a"}
Jul 07 11:42:31.325955 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:31.325584    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8" event={"ID":"7d8a3b4b-abb4-4dd0-ac55-407eaea08897","Type":"ContainerDied","Data":"11850440921fbb3291dcb08e7d9e629621f78aa1ca1ed3fd7455936fd0a7f323"}
Jul 07 11:42:31.325955 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:31.325617    2582 scope.go:117] "RemoveContainer" containerID="c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a"
Jul 07 11:42:31.325955 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:31.325879    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh" podUID="8ee6b677-5b10-4179-b799-d3d984c25a4d" containerName="istio-proxy" containerID="cri-o://122a49ff4725d4cf34264f71645b1660b969c8b609acc2e422536152818fc361" gracePeriod=30
Jul 07 11:42:31.338009 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:31.337991    2582 scope.go:117] "RemoveContainer" containerID="c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a"
Jul 07 11:42:31.338276 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:42:31.338256    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a\": container with ID starting with c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a not found: ID does not exist" containerID="c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a"
Jul 07 11:42:31.338353 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:31.338283    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a"} err="failed to get container status \"c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a\": rpc error: code = NotFound desc = could not find container \"c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a\": container with ID starting with c81683a8e4eefde1e25d2755fcc9837921ad41700c860b3e2d95c2064945716a not found: ID does not exist"
Jul 07 11:42:31.371221 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:31.362532    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"]
Jul 07 11:42:31.374954 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:31.374921    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-bdcf9e68-6d45464886-rxrv8"]
Jul 07 11:42:32.043454 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:32.043423    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:33.025251 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:33.025214    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d8a3b4b-abb4-4dd0-ac55-407eaea08897" path="/var/lib/kubelet/pods/7d8a3b4b-abb4-4dd0-ac55-407eaea08897/volumes"
Jul 07 11:42:36.351429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.351362    2582 generic.go:356] "Generic (PLEG): container finished" podID="8ee6b677-5b10-4179-b799-d3d984c25a4d" containerID="122a49ff4725d4cf34264f71645b1660b969c8b609acc2e422536152818fc361" exitCode=0
Jul 07 11:42:36.351429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.351415    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh" event={"ID":"8ee6b677-5b10-4179-b799-d3d984c25a4d","Type":"ContainerDied","Data":"122a49ff4725d4cf34264f71645b1660b969c8b609acc2e422536152818fc361"}
Jul 07 11:42:36.378097 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.378069    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:36.509010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.508982    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-envoy\") pod \"8ee6b677-5b10-4179-b799-d3d984c25a4d\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") "
Jul 07 11:42:36.509190 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509024    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkdcs\" (UniqueName: \"kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-kube-api-access-qkdcs\") pod \"8ee6b677-5b10-4179-b799-d3d984c25a4d\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") "
Jul 07 11:42:36.509190 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509048    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-data\") pod \"8ee6b677-5b10-4179-b799-d3d984c25a4d\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") "
Jul 07 11:42:36.509190 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509080    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ee6b677-5b10-4179-b799-d3d984c25a4d-istiod-ca-cert\") pod \"8ee6b677-5b10-4179-b799-d3d984c25a4d\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") "
Jul 07 11:42:36.509190 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509117    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-certs\") pod \"8ee6b677-5b10-4179-b799-d3d984c25a4d\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") "
Jul 07 11:42:36.509190 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509149    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-podinfo\") pod \"8ee6b677-5b10-4179-b799-d3d984c25a4d\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") "
Jul 07 11:42:36.509190 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509173    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-credential-socket\") pod \"8ee6b677-5b10-4179-b799-d3d984c25a4d\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") "
Jul 07 11:42:36.509953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509266    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-token\") pod \"8ee6b677-5b10-4179-b799-d3d984c25a4d\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") "
Jul 07 11:42:36.509953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509310    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-socket\") pod \"8ee6b677-5b10-4179-b799-d3d984c25a4d\" (UID: \"8ee6b677-5b10-4179-b799-d3d984c25a4d\") "
Jul 07 11:42:36.509953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509499    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ee6b677-5b10-4179-b799-d3d984c25a4d" (UID: "8ee6b677-5b10-4179-b799-d3d984c25a4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:42:36.509953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509507    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ee6b677-5b10-4179-b799-d3d984c25a4d" (UID: "8ee6b677-5b10-4179-b799-d3d984c25a4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:42:36.509953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509597    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-credential-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:36.509953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509614    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:36.509953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509700    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ee6b677-5b10-4179-b799-d3d984c25a4d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ee6b677-5b10-4179-b799-d3d984c25a4d" (UID: "8ee6b677-5b10-4179-b799-d3d984c25a4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:42:36.509953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.509861    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ee6b677-5b10-4179-b799-d3d984c25a4d" (UID: "8ee6b677-5b10-4179-b799-d3d984c25a4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:42:36.510292 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.510215    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ee6b677-5b10-4179-b799-d3d984c25a4d" (UID: "8ee6b677-5b10-4179-b799-d3d984c25a4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:42:36.511653 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.511618    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ee6b677-5b10-4179-b799-d3d984c25a4d" (UID: "8ee6b677-5b10-4179-b799-d3d984c25a4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:42:36.511653 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.511631    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ee6b677-5b10-4179-b799-d3d984c25a4d" (UID: "8ee6b677-5b10-4179-b799-d3d984c25a4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:42:36.511653 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.511635    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ee6b677-5b10-4179-b799-d3d984c25a4d" (UID: "8ee6b677-5b10-4179-b799-d3d984c25a4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:42:36.511924 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.511670    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-kube-api-access-qkdcs" (OuterVolumeSpecName: "kube-api-access-qkdcs") pod "8ee6b677-5b10-4179-b799-d3d984c25a4d" (UID: "8ee6b677-5b10-4179-b799-d3d984c25a4d"). InnerVolumeSpecName "kube-api-access-qkdcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:42:36.602026 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.601996    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:36.610132 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.610114    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ee6b677-5b10-4179-b799-d3d984c25a4d-istiod-ca-cert\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:36.610132 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.610134    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-podinfo\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:36.610289 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.610143    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-token\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:36.610289 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.610153    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-workload-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:36.610289 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.610162    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-envoy\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:36.610289 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.610170    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkdcs\" (UniqueName: \"kubernetes.io/projected/8ee6b677-5b10-4179-b799-d3d984c25a4d-kube-api-access-qkdcs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:36.610289 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.610178    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ee6b677-5b10-4179-b799-d3d984c25a4d-istio-data\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:42:36.707752 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:36.707719    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:37.078868 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:37.078836    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:37.174674 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:37.174645    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:37.277009 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:37.276979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:37.356753 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:37.356672    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh" event={"ID":"8ee6b677-5b10-4179-b799-d3d984c25a4d","Type":"ContainerDied","Data":"17770470a10bfb6e39fcc9074b25aa899cdbad94d4df10a54f76bbc44eac06a2"}
Jul 07 11:42:37.356753 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:37.356710    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"
Jul 07 11:42:37.356753 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:37.356734    2582 scope.go:117] "RemoveContainer" containerID="122a49ff4725d4cf34264f71645b1660b969c8b609acc2e422536152818fc361"
Jul 07 11:42:37.379905 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:37.379880    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"]
Jul 07 11:42:37.386603 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:37.386580    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-bdcf9e68-openshift-default-6694c98d6c-fjvfh"]
Jul 07 11:42:38.678833 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:38.678797    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:39.025389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:39.025359    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ee6b677-5b10-4179-b799-d3d984c25a4d" path="/var/lib/kubelet/pods/8ee6b677-5b10-4179-b799-d3d984c25a4d/volumes"
Jul 07 11:42:40.728519 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:40.728485    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:42:43.749745 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:43.749711    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:45.452218 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:45.452167    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:45.558458 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:45.558422    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:45.657773 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:45.657744    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:45.756001 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:45.755959    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:42:46.162739 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:42:46.162647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:12.198157 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:12.198105    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:18.145943 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:18.145891    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:18.256913 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:18.256885    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:18.353785 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:18.353753    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:34.300901 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:34.300863    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:40.536973 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:40.536942    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:40.642083 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:40.642049    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:40.733090 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:40.733057    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:51.259627 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:51.259549    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:52.111390 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:52.111354    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:58.020123 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:58.020084    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:58.157596 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:58.157555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:58.256065 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:58.256033    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:58.356112 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:58.356028    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:59.071831 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:59.071800    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:59.172109 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:59.172051    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:43:59.273499 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:43:59.273455    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:00.894748 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:00.894715    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:01.334397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:01.334363    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:44:07.279948 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:07.279912    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:07.374790 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:07.374764    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:07.948709 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:07.948664    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:09.816068 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:09.816033    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:44:13.933129 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:13.933087    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:14.033754 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.033721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:14.188390 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.188308    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:14.572241 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.572184    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"]
Jul 07 11:44:14.572721 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.572703    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d8a3b4b-abb4-4dd0-ac55-407eaea08897" containerName="maas-api"
Jul 07 11:44:14.572790 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.572725    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d8a3b4b-abb4-4dd0-ac55-407eaea08897" containerName="maas-api"
Jul 07 11:44:14.572790 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.572768    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ee6b677-5b10-4179-b799-d3d984c25a4d" containerName="istio-proxy"
Jul 07 11:44:14.572790 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.572778    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ee6b677-5b10-4179-b799-d3d984c25a4d" containerName="istio-proxy"
Jul 07 11:44:14.572922 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.572863    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d8a3b4b-abb4-4dd0-ac55-407eaea08897" containerName="maas-api"
Jul 07 11:44:14.572922 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.572879    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ee6b677-5b10-4179-b799-d3d984c25a4d" containerName="istio-proxy"
Jul 07 11:44:14.576403 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.576383    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.583973 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.583347    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 11:44:14.583973 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.583349    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-da2f4790-openshift-default-dockercfg-vkwbh\""
Jul 07 11:44:14.635324 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.635294    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"]
Jul 07 11:44:14.668102 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.668069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:14.686064 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.686031    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-659gz\" (UniqueName: \"kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-kube-api-access-659gz\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.686235 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.686075    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-credential-socket\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.686235 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.686102    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-envoy\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.686235 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.686185    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-token\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.686407 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.686243    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-data\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.686407 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.686294    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-socket\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.686407 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.686321    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/953c4761-2e96-40e9-b3eb-ec887b3e4672-istiod-ca-cert\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.686407 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.686351    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-podinfo\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.686589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.686431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-certs\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.787697 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.787666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-certs\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.787866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.787707    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-659gz\" (UniqueName: \"kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-kube-api-access-659gz\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.787866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.787737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-credential-socket\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.787866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.787759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-envoy\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.787866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.787817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-token\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.787866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.787845    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-data\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.788130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.787894    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-socket\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.788130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.787923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/953c4761-2e96-40e9-b3eb-ec887b3e4672-istiod-ca-cert\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.788130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.787952    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-podinfo\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.788130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.788070    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-certs\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.788387 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.788319    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-data\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.788387 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.788327    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-socket\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.788552 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.788527    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-credential-socket\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.788675 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.788659    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/953c4761-2e96-40e9-b3eb-ec887b3e4672-istiod-ca-cert\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.789933 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.789914    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-envoy\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.790255 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.790239    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-podinfo\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.800217 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.800170    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-token\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.800217 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.800184    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-659gz\" (UniqueName: \"kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-kube-api-access-659gz\") pod \"e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") " pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:14.846538 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.846467    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:14.887236 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:14.887216    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:15.028628 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:15.028597    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"]
Jul 07 11:44:15.031136 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:44:15.031112    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953c4761_2e96_40e9_b3eb_ec887b3e4672.slice/crio-01a7b7a1c89a76ca31773c1c981e3b06744c2e7566f5a6c6322034cbd3f7195b WatchSource:0}: Error finding container 01a7b7a1c89a76ca31773c1c981e3b06744c2e7566f5a6c6322034cbd3f7195b: Status 404 returned error can't find the container with id 01a7b7a1c89a76ca31773c1c981e3b06744c2e7566f5a6c6322034cbd3f7195b
Jul 07 11:44:15.032975 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:15.032940    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:44:15.033027 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:15.033004    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:44:15.033062 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:15.033033    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:44:15.811464 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:15.811424    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" event={"ID":"953c4761-2e96-40e9-b3eb-ec887b3e4672","Type":"ContainerStarted","Data":"1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62"}
Jul 07 11:44:15.811676 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:15.811522    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" event={"ID":"953c4761-2e96-40e9-b3eb-ec887b3e4672","Type":"ContainerStarted","Data":"01a7b7a1c89a76ca31773c1c981e3b06744c2e7566f5a6c6322034cbd3f7195b"}
Jul 07 11:44:15.835263 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:15.835178    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" podStartSLOduration=1.83516309 podStartE2EDuration="1.83516309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:44:15.832460451 +0000 UTC m=+1737.371151877" watchObservedRunningTime="2026-07-07 11:44:15.83516309 +0000 UTC m=+1737.373854501"
Jul 07 11:44:15.888065 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:15.888035    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:16.893185 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:16.893153    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:17.821151 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:17.821123    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:17.822023 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:17.822005    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:17.924316 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:17.924286    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"]
Jul 07 11:44:17.928175 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:17.928157    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:17.930752 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:17.930732    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-da2f4790-serving-cert\""
Jul 07 11:44:17.930876 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:17.930770    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-fpmsd\""
Jul 07 11:44:17.935634 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:17.935612    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"]
Jul 07 11:44:17.972654 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:17.972624    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:18.017826 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.017790    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-openshift-service-ca\") pod \"maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") " pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.017996 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.017843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbhdz\" (UniqueName: \"kubernetes.io/projected/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-kube-api-access-mbhdz\") pod \"maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") " pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.017996 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.017892    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-maas-api-tls\") pod \"maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") " pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.070846 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.070811    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:18.118762 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.118665    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-maas-api-tls\") pod \"maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") " pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.119049 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:44:18.119026    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-da2f4790-serving-cert: secret "maas-api-e2e-mt-da2f4790-serving-cert" not found
Jul 07 11:44:18.119152 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.119066    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-openshift-service-ca\") pod \"maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") " pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.119152 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:44:18.119097    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-maas-api-tls podName:cfd2cf5c-6da2-4777-a776-3daf2d3a7207 nodeName:}" failed. No retries permitted until 2026-07-07 11:44:18.619073411 +0000 UTC m=+1740.157764825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-maas-api-tls") pod "maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd" (UID: "cfd2cf5c-6da2-4777-a776-3daf2d3a7207") : secret "maas-api-e2e-mt-da2f4790-serving-cert" not found
Jul 07 11:44:18.119303 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.119249    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbhdz\" (UniqueName: \"kubernetes.io/projected/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-kube-api-access-mbhdz\") pod \"maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") " pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.119686 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.119665    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-openshift-service-ca\") pod \"maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") " pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.128553 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.128523    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbhdz\" (UniqueName: \"kubernetes.io/projected/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-kube-api-access-mbhdz\") pod \"maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") " pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.301400 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.301359    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:18.435124 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.435032    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:18.533463 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.533427    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:18.623508 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.623466    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-maas-api-tls\") pod \"maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") " pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.625880 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.625859    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-maas-api-tls\") pod \"maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") " pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.842761 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.842732    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:18.982298 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:18.982268    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"]
Jul 07 11:44:18.985667 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:44:18.985644    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd2cf5c_6da2_4777_a776_3daf2d3a7207.slice/crio-b90ce068e24683f062e774de65e73678b145d533929d82e92af6e4a1d062da03 WatchSource:0}: Error finding container b90ce068e24683f062e774de65e73678b145d533929d82e92af6e4a1d062da03: Status 404 returned error can't find the container with id b90ce068e24683f062e774de65e73678b145d533929d82e92af6e4a1d062da03
Jul 07 11:44:19.830626 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:19.830587    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd" event={"ID":"cfd2cf5c-6da2-4777-a776-3daf2d3a7207","Type":"ContainerStarted","Data":"12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b"}
Jul 07 11:44:19.830822 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:19.830639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd" event={"ID":"cfd2cf5c-6da2-4777-a776-3daf2d3a7207","Type":"ContainerStarted","Data":"b90ce068e24683f062e774de65e73678b145d533929d82e92af6e4a1d062da03"}
Jul 07 11:44:19.830822 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:19.830665    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:19.852413 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:19.852370    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd" podStartSLOduration=2.453431053 podStartE2EDuration="2.852356139s" totalImagesPullingTime="398.925086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:44:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:44:19.850016337 +0000 UTC m=+1741.388707750" watchObservedRunningTime="2026-07-07 11:44:19.852356139 +0000 UTC m=+1741.391047550"
Jul 07 11:44:22.382662 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:22.382625    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:23.028328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:23.028284    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:23.127013 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:23.126983    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:23.472179 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:23.472151    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:24.022635 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:24.022603    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:44:25.841387 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:25.841356    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:29.074692 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:29.074656    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:29.175234 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:29.175180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:29.498824 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:29.498787    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:29.790460 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:29.790375    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"]
Jul 07 11:44:29.790654 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:29.790613    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd" podUID="cfd2cf5c-6da2-4777-a776-3daf2d3a7207" containerName="maas-api" containerID="cri-o://12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b" gracePeriod=30
Jul 07 11:44:29.816240 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:29.816189    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:29.883652 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:29.883614    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:30.078966 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.078935    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:30.240975 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.240936    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-maas-api-tls\") pod \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") "
Jul 07 11:44:30.241132 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.241015    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbhdz\" (UniqueName: \"kubernetes.io/projected/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-kube-api-access-mbhdz\") pod \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") "
Jul 07 11:44:30.241132 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.241123    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-openshift-service-ca\") pod \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\" (UID: \"cfd2cf5c-6da2-4777-a776-3daf2d3a7207\") "
Jul 07 11:44:30.241514 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.241481    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cfd2cf5c-6da2-4777-a776-3daf2d3a7207" (UID: "cfd2cf5c-6da2-4777-a776-3daf2d3a7207"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:44:30.243233 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.243186    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cfd2cf5c-6da2-4777-a776-3daf2d3a7207" (UID: "cfd2cf5c-6da2-4777-a776-3daf2d3a7207"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:44:30.243475 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.243456    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-kube-api-access-mbhdz" (OuterVolumeSpecName: "kube-api-access-mbhdz") pod "cfd2cf5c-6da2-4777-a776-3daf2d3a7207" (UID: "cfd2cf5c-6da2-4777-a776-3daf2d3a7207"). InnerVolumeSpecName "kube-api-access-mbhdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:44:30.342511 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.342419    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:30.342511 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.342464    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:30.342511 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.342480    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbhdz\" (UniqueName: \"kubernetes.io/projected/cfd2cf5c-6da2-4777-a776-3daf2d3a7207-kube-api-access-mbhdz\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:30.886772 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.886740    2582 generic.go:356] "Generic (PLEG): container finished" podID="cfd2cf5c-6da2-4777-a776-3daf2d3a7207" containerID="12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b" exitCode=0
Jul 07 11:44:30.886979 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.886795    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"
Jul 07 11:44:30.886979 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.886825    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd" event={"ID":"cfd2cf5c-6da2-4777-a776-3daf2d3a7207","Type":"ContainerDied","Data":"12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b"}
Jul 07 11:44:30.886979 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.886881    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd" event={"ID":"cfd2cf5c-6da2-4777-a776-3daf2d3a7207","Type":"ContainerDied","Data":"b90ce068e24683f062e774de65e73678b145d533929d82e92af6e4a1d062da03"}
Jul 07 11:44:30.886979 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.886903    2582 scope.go:117] "RemoveContainer" containerID="12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b"
Jul 07 11:44:30.896400 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.896382    2582 scope.go:117] "RemoveContainer" containerID="12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b"
Jul 07 11:44:30.896662 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:44:30.896640    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b\": container with ID starting with 12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b not found: ID does not exist" containerID="12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b"
Jul 07 11:44:30.896728 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.896673    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b"} err="failed to get container status \"12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b\": rpc error: code = NotFound desc = could not find container \"12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b\": container with ID starting with 12940af88791cd7927f5a099ff5a90465531fe5319a54515d32c09d2b932839b not found: ID does not exist"
Jul 07 11:44:30.909052 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.909028    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"]
Jul 07 11:44:30.914766 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:30.914743    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-da2f4790-5b894b55cb-2mztd"]
Jul 07 11:44:31.025051 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:31.025018    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfd2cf5c-6da2-4777-a776-3daf2d3a7207" path="/var/lib/kubelet/pods/cfd2cf5c-6da2-4777-a776-3daf2d3a7207/volumes"
Jul 07 11:44:31.892825 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:31.892756    2582 generic.go:356] "Generic (PLEG): container finished" podID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerID="1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62" exitCode=0
Jul 07 11:44:31.893261 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:31.892828    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" event={"ID":"953c4761-2e96-40e9-b3eb-ec887b3e4672","Type":"ContainerDied","Data":"1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62"}
Jul 07 11:44:31.893352 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:31.893297    2582 scope.go:117] "RemoveContainer" containerID="1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62"
Jul 07 11:44:31.894315 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:31.894282    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:44:31.894397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:31.894353    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:44:31.894397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:31.894388    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:44:32.624647 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:32.624616    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:44:32.821840 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:32.821812    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:32.900397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:32.900316    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" event={"ID":"953c4761-2e96-40e9-b3eb-ec887b3e4672","Type":"ContainerStarted","Data":"64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e"}
Jul 07 11:44:33.888385 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:33.888349    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:33.889654 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:33.889620    2582 patch_prober.go:28] interesting pod/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 07 11:44:33.889807 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:33.889678    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 07 11:44:34.888135 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:34.888099    2582 patch_prober.go:28] interesting pod/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 07 11:44:34.888755 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:34.888708    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 07 11:44:35.887981 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:35.887941    2582 patch_prober.go:28] interesting pod/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 07 11:44:35.888163 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:35.888010    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 07 11:44:36.601245 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:36.601213    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:36.747801 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:36.747767    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:36.888454 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:36.888367    2582 patch_prober.go:28] interesting pod/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 07 11:44:36.889513 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:36.888991    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 07 11:44:36.944896 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:36.944862    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"]
Jul 07 11:44:36.945104 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:36.945055    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy" containerID="cri-o://64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e" gracePeriod=30
Jul 07 11:44:36.988022 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:36.987987    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:37.165675 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:37.165539    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:37.294293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:37.294232    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:37.394638 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:37.394603    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:37.526230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:37.526180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:42.288486 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.288460    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:42.356316 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356247    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-podinfo\") pod \"953c4761-2e96-40e9-b3eb-ec887b3e4672\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") "
Jul 07 11:44:42.356437 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356331    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-token\") pod \"953c4761-2e96-40e9-b3eb-ec887b3e4672\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") "
Jul 07 11:44:42.356437 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356377    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-credential-socket\") pod \"953c4761-2e96-40e9-b3eb-ec887b3e4672\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") "
Jul 07 11:44:42.356437 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356400    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-data\") pod \"953c4761-2e96-40e9-b3eb-ec887b3e4672\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") "
Jul 07 11:44:42.356437 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356429    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-socket\") pod \"953c4761-2e96-40e9-b3eb-ec887b3e4672\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") "
Jul 07 11:44:42.356635 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356447    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-659gz\" (UniqueName: \"kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-kube-api-access-659gz\") pod \"953c4761-2e96-40e9-b3eb-ec887b3e4672\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") "
Jul 07 11:44:42.356635 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356464    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-certs\") pod \"953c4761-2e96-40e9-b3eb-ec887b3e4672\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") "
Jul 07 11:44:42.356635 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356490    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/953c4761-2e96-40e9-b3eb-ec887b3e4672-istiod-ca-cert\") pod \"953c4761-2e96-40e9-b3eb-ec887b3e4672\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") "
Jul 07 11:44:42.356635 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356521    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-envoy\") pod \"953c4761-2e96-40e9-b3eb-ec887b3e4672\" (UID: \"953c4761-2e96-40e9-b3eb-ec887b3e4672\") "
Jul 07 11:44:42.356850 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356675    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "953c4761-2e96-40e9-b3eb-ec887b3e4672" (UID: "953c4761-2e96-40e9-b3eb-ec887b3e4672"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:44:42.356850 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356708    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "953c4761-2e96-40e9-b3eb-ec887b3e4672" (UID: "953c4761-2e96-40e9-b3eb-ec887b3e4672"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:44:42.356850 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356722    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "953c4761-2e96-40e9-b3eb-ec887b3e4672" (UID: "953c4761-2e96-40e9-b3eb-ec887b3e4672"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:44:42.356850 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356802    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-credential-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:42.356850 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356821    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:42.356850 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.356834    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-workload-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:42.357130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.357017    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/953c4761-2e96-40e9-b3eb-ec887b3e4672-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "953c4761-2e96-40e9-b3eb-ec887b3e4672" (UID: "953c4761-2e96-40e9-b3eb-ec887b3e4672"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:44:42.357395 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.357372    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-data" (OuterVolumeSpecName: "istio-data") pod "953c4761-2e96-40e9-b3eb-ec887b3e4672" (UID: "953c4761-2e96-40e9-b3eb-ec887b3e4672"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:44:42.358706 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.358678    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-kube-api-access-659gz" (OuterVolumeSpecName: "kube-api-access-659gz") pod "953c4761-2e96-40e9-b3eb-ec887b3e4672" (UID: "953c4761-2e96-40e9-b3eb-ec887b3e4672"). InnerVolumeSpecName "kube-api-access-659gz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:44:42.358817 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.358765    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "953c4761-2e96-40e9-b3eb-ec887b3e4672" (UID: "953c4761-2e96-40e9-b3eb-ec887b3e4672"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:44:42.358952 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.358933    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-token" (OuterVolumeSpecName: "istio-token") pod "953c4761-2e96-40e9-b3eb-ec887b3e4672" (UID: "953c4761-2e96-40e9-b3eb-ec887b3e4672"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:44:42.359027 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.358954    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "953c4761-2e96-40e9-b3eb-ec887b3e4672" (UID: "953c4761-2e96-40e9-b3eb-ec887b3e4672"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:44:42.457423 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.457386    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-659gz\" (UniqueName: \"kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-kube-api-access-659gz\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:42.457423 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.457421    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/953c4761-2e96-40e9-b3eb-ec887b3e4672-istiod-ca-cert\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:42.457423 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.457431    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-envoy\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:42.457612 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.457439    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-podinfo\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:42.457612 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.457446    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-token\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:42.457612 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.457454    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/953c4761-2e96-40e9-b3eb-ec887b3e4672-istio-data\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:44:42.949991 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.949955    2582 generic.go:356] "Generic (PLEG): container finished" podID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerID="64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e" exitCode=0
Jul 07 11:44:42.950156 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.950008    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"
Jul 07 11:44:42.950156 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.950040    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" event={"ID":"953c4761-2e96-40e9-b3eb-ec887b3e4672","Type":"ContainerDied","Data":"64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e"}
Jul 07 11:44:42.950156 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.950094    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k" event={"ID":"953c4761-2e96-40e9-b3eb-ec887b3e4672","Type":"ContainerDied","Data":"01a7b7a1c89a76ca31773c1c981e3b06744c2e7566f5a6c6322034cbd3f7195b"}
Jul 07 11:44:42.950156 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.950125    2582 scope.go:117] "RemoveContainer" containerID="64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e"
Jul 07 11:44:42.959696 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.959679    2582 scope.go:117] "RemoveContainer" containerID="1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62"
Jul 07 11:44:42.967460 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.967444    2582 scope.go:117] "RemoveContainer" containerID="64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e"
Jul 07 11:44:42.967697 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:44:42.967678    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e\": container with ID starting with 64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e not found: ID does not exist" containerID="64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e"
Jul 07 11:44:42.967777 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.967710    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e"} err="failed to get container status \"64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e\": rpc error: code = NotFound desc = could not find container \"64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e\": container with ID starting with 64625bb1dfd03b37703f4aa7c279b3fce4a8e4bf479713788242533d5f9afa4e not found: ID does not exist"
Jul 07 11:44:42.967777 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.967736    2582 scope.go:117] "RemoveContainer" containerID="1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62"
Jul 07 11:44:42.967975 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:44:42.967956    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62\": container with ID starting with 1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62 not found: ID does not exist" containerID="1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62"
Jul 07 11:44:42.968016 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.967982    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62"} err="failed to get container status \"1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62\": rpc error: code = NotFound desc = could not find container \"1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62\": container with ID starting with 1c431c46aea73bbb89dfd60565ab9c093315acf72ff32f49590fc77efa1f9d62 not found: ID does not exist"
Jul 07 11:44:42.972660 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.972639    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"]
Jul 07 11:44:42.977015 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:42.976995    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-da2f4790-openshift-default-669b7d96f9-h6n2k"]
Jul 07 11:44:43.025054 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:43.025024    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" path="/var/lib/kubelet/pods/953c4761-2e96-40e9-b3eb-ec887b3e4672/volumes"
Jul 07 11:44:43.301509 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:43.301479    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:43.403759 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:43.403727    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:43.547917 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:43.547883    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:43.642450 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:43.642369    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:43.742937 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:43.742907    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:50.705757 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:50.705723    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:50.854209 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:50.854164    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:57.569639 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.569586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:57.839458 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.839378    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"]
Jul 07 11:44:57.839720 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.839708    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy"
Jul 07 11:44:57.839763 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.839721    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy"
Jul 07 11:44:57.839763 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.839754    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd2cf5c-6da2-4777-a776-3daf2d3a7207" containerName="maas-api"
Jul 07 11:44:57.839763 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.839762    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd2cf5c-6da2-4777-a776-3daf2d3a7207" containerName="maas-api"
Jul 07 11:44:57.839858 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.839819    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy"
Jul 07 11:44:57.839858 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.839826    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd2cf5c-6da2-4777-a776-3daf2d3a7207" containerName="maas-api"
Jul 07 11:44:57.839858 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.839832    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy"
Jul 07 11:44:57.842800 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.842782    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.845307 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.845283    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 11:44:57.845421 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.845368    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-23c41a78-openshift-default-dockercfg-tslpk\""
Jul 07 11:44:57.854944 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.854920    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"]
Jul 07 11:44:57.861223 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.861157    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:57.883684 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.883659    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-certs\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.883777 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.883719    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-credential-socket\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.883849 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.883779    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-socket\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.883849 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.883807    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa609900-0ee5-4ce3-858e-885cd97c3e58-istiod-ca-cert\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.883938 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.883847    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pblbh\" (UniqueName: \"kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-kube-api-access-pblbh\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.883970 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.883939    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-data\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.884001 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.883972    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-envoy\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.884001 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.883989    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-podinfo\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.884073 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.884010    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-token\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985136 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985103    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pblbh\" (UniqueName: \"kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-kube-api-access-pblbh\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985333 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-data\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985333 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985176    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-envoy\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985333 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985216    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-podinfo\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985333 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985240    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-token\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985333 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985272    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-certs\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985333 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985328    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-credential-socket\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985688 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985359    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-socket\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985688 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985384    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa609900-0ee5-4ce3-858e-885cd97c3e58-istiod-ca-cert\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985688 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985521    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-data\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.985967 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985943    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-certs\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.986044 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.985971    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-socket\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.986100 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.986064    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa609900-0ee5-4ce3-858e-885cd97c3e58-istiod-ca-cert\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.986100 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.986077    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-credential-socket\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.987502 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.987482    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-envoy\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.987872 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.987854    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-podinfo\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.993355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.993329    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pblbh\" (UniqueName: \"kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-kube-api-access-pblbh\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:57.993440 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:57.993386    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-token\") pod \"e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") " pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:58.153145 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:58.153045    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:58.291882 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:58.291854    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"]
Jul 07 11:44:58.293156 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:44:58.293129    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa609900_0ee5_4ce3_858e_885cd97c3e58.slice/crio-0aa2b193a235fa50a069a8ba96c8ab5804670fc17166fd5c800d7bc17e13527f WatchSource:0}: Error finding container 0aa2b193a235fa50a069a8ba96c8ab5804670fc17166fd5c800d7bc17e13527f: Status 404 returned error can't find the container with id 0aa2b193a235fa50a069a8ba96c8ab5804670fc17166fd5c800d7bc17e13527f
Jul 07 11:44:58.295272 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:58.295235    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:44:58.295350 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:58.295306    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:44:58.295350 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:58.295333    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:44:58.521034 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:58.520996    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:44:59.025176 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:59.025141    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" event={"ID":"fa609900-0ee5-4ce3-858e-885cd97c3e58","Type":"ContainerStarted","Data":"a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e"}
Jul 07 11:44:59.025654 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:59.025214    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" event={"ID":"fa609900-0ee5-4ce3-858e-885cd97c3e58","Type":"ContainerStarted","Data":"0aa2b193a235fa50a069a8ba96c8ab5804670fc17166fd5c800d7bc17e13527f"}
Jul 07 11:44:59.046078 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:59.046018    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" podStartSLOduration=2.045999242 podStartE2EDuration="2.045999242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:44:59.045838182 +0000 UTC m=+1780.584529608" watchObservedRunningTime="2026-07-07 11:44:59.045999242 +0000 UTC m=+1780.584690654"
Jul 07 11:44:59.153807 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:59.153777    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:44:59.165102 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:44:59.165074    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:45:00.029640 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.029609    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:45:00.030589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.030571    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:45:00.128899 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.128871    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723745-lclc5"]
Jul 07 11:45:00.134822 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.134788    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy"
Jul 07 11:45:00.134822 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.134821    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="953c4761-2e96-40e9-b3eb-ec887b3e4672" containerName="istio-proxy"
Jul 07 11:45:00.138752 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.138723    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723745-lclc5"
Jul 07 11:45:00.140876 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.140849    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723745-lclc5"]
Jul 07 11:45:00.141470 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.141448    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-fpmsd\""
Jul 07 11:45:00.205336 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.205314    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbzd6\" (UniqueName: \"kubernetes.io/projected/0df681bd-2339-4c56-92a0-7d29c68724fd-kube-api-access-xbzd6\") pod \"maas-api-key-cleanup-29723745-lclc5\" (UID: \"0df681bd-2339-4c56-92a0-7d29c68724fd\") " pod="opendatahub/maas-api-key-cleanup-29723745-lclc5"
Jul 07 11:45:00.306544 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.306485    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbzd6\" (UniqueName: \"kubernetes.io/projected/0df681bd-2339-4c56-92a0-7d29c68724fd-kube-api-access-xbzd6\") pod \"maas-api-key-cleanup-29723745-lclc5\" (UID: \"0df681bd-2339-4c56-92a0-7d29c68724fd\") " pod="opendatahub/maas-api-key-cleanup-29723745-lclc5"
Jul 07 11:45:00.316155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.316132    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbzd6\" (UniqueName: \"kubernetes.io/projected/0df681bd-2339-4c56-92a0-7d29c68724fd-kube-api-access-xbzd6\") pod \"maas-api-key-cleanup-29723745-lclc5\" (UID: \"0df681bd-2339-4c56-92a0-7d29c68724fd\") " pod="opendatahub/maas-api-key-cleanup-29723745-lclc5"
Jul 07 11:45:00.450748 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.450724    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723745-lclc5"
Jul 07 11:45:00.583855 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:00.583828    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723745-lclc5"]
Jul 07 11:45:00.586111 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:45:00.586089    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df681bd_2339_4c56_92a0_7d29c68724fd.slice/crio-6587168156a4eb24296a41861a965b2743676b17c874398b053e7604c0d9ac06 WatchSource:0}: Error finding container 6587168156a4eb24296a41861a965b2743676b17c874398b053e7604c0d9ac06: Status 404 returned error can't find the container with id 6587168156a4eb24296a41861a965b2743676b17c874398b053e7604c0d9ac06
Jul 07 11:45:01.034496 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.034451    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723745-lclc5" event={"ID":"0df681bd-2339-4c56-92a0-7d29c68724fd","Type":"ContainerStarted","Data":"3a82d6780edaef1e3cf2af4ce774d036bedb13e10cdc5792f87addae3c2923a0"}
Jul 07 11:45:01.034913 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.034510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723745-lclc5" event={"ID":"0df681bd-2339-4c56-92a0-7d29c68724fd","Type":"ContainerStarted","Data":"6587168156a4eb24296a41861a965b2743676b17c874398b053e7604c0d9ac06"}
Jul 07 11:45:01.046823 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.046792    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:45:01.062318 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.062273    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723745-lclc5" podStartSLOduration=1.062185184 podStartE2EDuration="1.062185184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:45:01.059855493 +0000 UTC m=+1782.598546916" watchObservedRunningTime="2026-07-07 11:45:01.062185184 +0000 UTC m=+1782.600876595"
Jul 07 11:45:01.106927 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.106899    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"]
Jul 07 11:45:01.110234 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.110213    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.112851 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.112834    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-23c41a78-serving-cert\""
Jul 07 11:45:01.122791 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.122767    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"]
Jul 07 11:45:01.133783 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.133750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:45:01.214058 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.214023    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63fdfe2d-d11f-452a-a507-b9dc916db51a-openshift-service-ca\") pod \"maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") " pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.214255 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.214109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4qwh\" (UniqueName: \"kubernetes.io/projected/63fdfe2d-d11f-452a-a507-b9dc916db51a-kube-api-access-t4qwh\") pod \"maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") " pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.214255 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.214175    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63fdfe2d-d11f-452a-a507-b9dc916db51a-maas-api-tls\") pod \"maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") " pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.227030 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.227000    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:45:01.315495 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.315413    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63fdfe2d-d11f-452a-a507-b9dc916db51a-maas-api-tls\") pod \"maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") " pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.315651 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.315502    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63fdfe2d-d11f-452a-a507-b9dc916db51a-openshift-service-ca\") pod \"maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") " pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.315651 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.315558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4qwh\" (UniqueName: \"kubernetes.io/projected/63fdfe2d-d11f-452a-a507-b9dc916db51a-kube-api-access-t4qwh\") pod \"maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") " pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.315651 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:45:01.315561    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-23c41a78-serving-cert: secret "maas-api-e2e-mt-23c41a78-serving-cert" not found
Jul 07 11:45:01.315651 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:45:01.315622    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63fdfe2d-d11f-452a-a507-b9dc916db51a-maas-api-tls podName:63fdfe2d-d11f-452a-a507-b9dc916db51a nodeName:}" failed. No retries permitted until 2026-07-07 11:45:01.815606133 +0000 UTC m=+1783.354297522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63fdfe2d-d11f-452a-a507-b9dc916db51a-maas-api-tls") pod "maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt" (UID: "63fdfe2d-d11f-452a-a507-b9dc916db51a") : secret "maas-api-e2e-mt-23c41a78-serving-cert" not found
Jul 07 11:45:01.316110 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.316090    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63fdfe2d-d11f-452a-a507-b9dc916db51a-openshift-service-ca\") pod \"maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") " pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.330365 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.330338    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4qwh\" (UniqueName: \"kubernetes.io/projected/63fdfe2d-d11f-452a-a507-b9dc916db51a-kube-api-access-t4qwh\") pod \"maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") " pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.430840 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.430810    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:45:01.810132 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.810104    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:45:01.819280 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.819256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63fdfe2d-d11f-452a-a507-b9dc916db51a-maas-api-tls\") pod \"maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") " pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.821793 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.821739    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63fdfe2d-d11f-452a-a507-b9dc916db51a-maas-api-tls\") pod \"maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") " pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:01.910161 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:01.910126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:45:02.022299 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:02.022267    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:02.041403 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:02.041372    2582 generic.go:356] "Generic (PLEG): container finished" podID="0df681bd-2339-4c56-92a0-7d29c68724fd" containerID="3a82d6780edaef1e3cf2af4ce774d036bedb13e10cdc5792f87addae3c2923a0" exitCode=0
Jul 07 11:45:02.041718 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:02.041456    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723745-lclc5" event={"ID":"0df681bd-2339-4c56-92a0-7d29c68724fd","Type":"ContainerDied","Data":"3a82d6780edaef1e3cf2af4ce774d036bedb13e10cdc5792f87addae3c2923a0"}
Jul 07 11:45:02.156032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:02.156008    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"]
Jul 07 11:45:02.157839 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:45:02.157813    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63fdfe2d_d11f_452a_a507_b9dc916db51a.slice/crio-aa0214a7bc61e73db20057b8f89b7c94e4f17c869e7d98a2ab4aee6ad8a5ffca WatchSource:0}: Error finding container aa0214a7bc61e73db20057b8f89b7c94e4f17c869e7d98a2ab4aee6ad8a5ffca: Status 404 returned error can't find the container with id aa0214a7bc61e73db20057b8f89b7c94e4f17c869e7d98a2ab4aee6ad8a5ffca
Jul 07 11:45:03.046542 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:03.046503    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt" event={"ID":"63fdfe2d-d11f-452a-a507-b9dc916db51a","Type":"ContainerStarted","Data":"aa0214a7bc61e73db20057b8f89b7c94e4f17c869e7d98a2ab4aee6ad8a5ffca"}
Jul 07 11:45:03.184526 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:03.184499    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723745-lclc5"
Jul 07 11:45:03.231474 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:03.231437    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbzd6\" (UniqueName: \"kubernetes.io/projected/0df681bd-2339-4c56-92a0-7d29c68724fd-kube-api-access-xbzd6\") pod \"0df681bd-2339-4c56-92a0-7d29c68724fd\" (UID: \"0df681bd-2339-4c56-92a0-7d29c68724fd\") "
Jul 07 11:45:03.233440 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:03.233414    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0df681bd-2339-4c56-92a0-7d29c68724fd-kube-api-access-xbzd6" (OuterVolumeSpecName: "kube-api-access-xbzd6") pod "0df681bd-2339-4c56-92a0-7d29c68724fd" (UID: "0df681bd-2339-4c56-92a0-7d29c68724fd"). InnerVolumeSpecName "kube-api-access-xbzd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:45:03.332666 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:03.332647    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbzd6\" (UniqueName: \"kubernetes.io/projected/0df681bd-2339-4c56-92a0-7d29c68724fd-kube-api-access-xbzd6\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:45:04.052948 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:04.052907    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt" event={"ID":"63fdfe2d-d11f-452a-a507-b9dc916db51a","Type":"ContainerStarted","Data":"11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b"}
Jul 07 11:45:04.053432 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:04.052964    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:04.054160 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:04.054141    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723745-lclc5"
Jul 07 11:45:04.054278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:04.054155    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723745-lclc5" event={"ID":"0df681bd-2339-4c56-92a0-7d29c68724fd","Type":"ContainerDied","Data":"6587168156a4eb24296a41861a965b2743676b17c874398b053e7604c0d9ac06"}
Jul 07 11:45:04.054278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:04.054211    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6587168156a4eb24296a41861a965b2743676b17c874398b053e7604c0d9ac06"
Jul 07 11:45:04.085590 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:04.085544    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt" podStartSLOduration=1.963246069 podStartE2EDuration="3.085527354s" totalImagesPullingTime="1.122281285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:45:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:45:04.084648598 +0000 UTC m=+1785.623340012" watchObservedRunningTime="2026-07-07 11:45:04.085527354 +0000 UTC m=+1785.624218769"
Jul 07 11:45:05.547163 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:05.547130    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:45:05.997004 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:05.996971    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:45:10.069477 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:10.069445    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:14.113445 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:14.113409    2582 generic.go:356] "Generic (PLEG): container finished" podID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerID="a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e" exitCode=0
Jul 07 11:45:14.113893 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:14.113460    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" event={"ID":"fa609900-0ee5-4ce3-858e-885cd97c3e58","Type":"ContainerDied","Data":"a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e"}
Jul 07 11:45:14.113893 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:14.113802    2582 scope.go:117] "RemoveContainer" containerID="a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e"
Jul 07 11:45:14.114738 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:14.114706    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:45:14.114855 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:14.114777    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:45:14.114855 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:14.114809    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:45:15.030000 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.029972    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:45:15.120261 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.120221    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" event={"ID":"fa609900-0ee5-4ce3-858e-885cd97c3e58","Type":"ContainerStarted","Data":"08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c"}
Jul 07 11:45:15.153311 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.153289    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:45:15.154227 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.154183    2582 patch_prober.go:28] interesting pod/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 07 11:45:15.154334 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.154248    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 07 11:45:15.900315 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.900281    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"]
Jul 07 11:45:15.900703 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.900686    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0df681bd-2339-4c56-92a0-7d29c68724fd" containerName="cleanup"
Jul 07 11:45:15.900789 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.900706    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0df681bd-2339-4c56-92a0-7d29c68724fd" containerName="cleanup"
Jul 07 11:45:15.900845 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.900792    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0df681bd-2339-4c56-92a0-7d29c68724fd" containerName="cleanup"
Jul 07 11:45:15.904595 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.904571    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:15.907306 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.907284    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-23c41a78\"/\"openshift-service-ca.crt\""
Jul 07 11:45:15.908414 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.908396    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-23c41a78\"/\"default-dockercfg-v7bqn\""
Jul 07 11:45:15.908527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.908446    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-23c41a78\"/\"kube-root-ca.crt\""
Jul 07 11:45:15.908527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.908458    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-23c41a78\"/\"e2e-lifecycle-model-23c41a78-kserve-self-signed-certs\""
Jul 07 11:45:15.921564 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.921542    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"]
Jul 07 11:45:15.979613 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:15.979583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:45:16.051116 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.051085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tls-certs\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.051273 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.051123    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-home\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.051319 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.051267    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-dshm\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.051319 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.051300    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-model-cache\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.051391 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.051321    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2c8f\" (UniqueName: \"kubernetes.io/projected/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-kube-api-access-l2c8f\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.051391 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.051353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tmp-dir\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.152734 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.152641    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-dshm\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.152734 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.152676    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-model-cache\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.152734 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.152695    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2c8f\" (UniqueName: \"kubernetes.io/projected/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-kube-api-access-l2c8f\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.153293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.152872    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tmp-dir\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.153293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.152924    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tls-certs\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.153293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.152955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-home\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.153293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.153179    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-model-cache\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.153293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.153250    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-home\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.153470 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.153290    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tmp-dir\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.153571 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.153548    2582 patch_prober.go:28] interesting pod/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 07 11:45:16.153624 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.153603    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 07 11:45:16.154988 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.154966    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-dshm\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.155348 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.155329    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tls-certs\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.161801 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.161780    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2c8f\" (UniqueName: \"kubernetes.io/projected/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-kube-api-access-l2c8f\") pod \"e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") " pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.214630 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.214601    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:16.390635 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:16.390604    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"]
Jul 07 11:45:16.393997 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:45:16.393966    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaeeea2c9_53a9_46f1_a321_2d3dca6b19eb.slice/crio-eeac667a5013b4c7af238401e1b66935eb2beb6fa3a9e3c1fe678941cbacb7bb WatchSource:0}: Error finding container eeac667a5013b4c7af238401e1b66935eb2beb6fa3a9e3c1fe678941cbacb7bb: Status 404 returned error can't find the container with id eeac667a5013b4c7af238401e1b66935eb2beb6fa3a9e3c1fe678941cbacb7bb
Jul 07 11:45:17.139315 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:17.139287    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw" event={"ID":"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb","Type":"ContainerStarted","Data":"07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a"}
Jul 07 11:45:17.139435 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:17.139330    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw" event={"ID":"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb","Type":"ContainerStarted","Data":"eeac667a5013b4c7af238401e1b66935eb2beb6fa3a9e3c1fe678941cbacb7bb"}
Jul 07 11:45:17.153917 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:17.153889    2582 patch_prober.go:28] interesting pod/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 07 11:45:17.154219 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:17.153950    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 07 11:45:17.187232 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:17.182710    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw" podStartSLOduration=2.182690216 podStartE2EDuration="2.182690216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:45:17.181563012 +0000 UTC m=+1798.720254425" watchObservedRunningTime="2026-07-07 11:45:17.182690216 +0000 UTC m=+1798.721381629"
Jul 07 11:45:18.153832 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:18.153798    2582 patch_prober.go:28] interesting pod/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 07 11:45:18.154008 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:18.153849    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 07 11:45:19.031015 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:19.030990    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:45:19.033626 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:19.033601    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:45:19.157072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:19.157044    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:45:19.157496 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:19.157471    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:45:19.158158 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:19.158140    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:45:20.027978 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:20.027943    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:45:26.214784 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:26.214745    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:26.215307 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:26.214929    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:26.227837 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:26.227815    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:27.199234 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:27.199185    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:45:47.526764 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:47.526727    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:45:53.391734 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:53.391687    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:45:53.718705 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:53.718672    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"]
Jul 07 11:45:53.719154 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:53.719120    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt" podUID="63fdfe2d-d11f-452a-a507-b9dc916db51a" containerName="maas-api" containerID="cri-o://11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b" gracePeriod=30
Jul 07 11:45:53.763558 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:53.763528    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:45:53.973607 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:53.973547    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:54.101160 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.101134    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4qwh\" (UniqueName: \"kubernetes.io/projected/63fdfe2d-d11f-452a-a507-b9dc916db51a-kube-api-access-t4qwh\") pod \"63fdfe2d-d11f-452a-a507-b9dc916db51a\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") "
Jul 07 11:45:54.101363 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.101172    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63fdfe2d-d11f-452a-a507-b9dc916db51a-openshift-service-ca\") pod \"63fdfe2d-d11f-452a-a507-b9dc916db51a\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") "
Jul 07 11:45:54.101363 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.101223    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63fdfe2d-d11f-452a-a507-b9dc916db51a-maas-api-tls\") pod \"63fdfe2d-d11f-452a-a507-b9dc916db51a\" (UID: \"63fdfe2d-d11f-452a-a507-b9dc916db51a\") "
Jul 07 11:45:54.101640 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.101607    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63fdfe2d-d11f-452a-a507-b9dc916db51a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63fdfe2d-d11f-452a-a507-b9dc916db51a" (UID: "63fdfe2d-d11f-452a-a507-b9dc916db51a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:45:54.103350 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.103327    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63fdfe2d-d11f-452a-a507-b9dc916db51a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63fdfe2d-d11f-452a-a507-b9dc916db51a" (UID: "63fdfe2d-d11f-452a-a507-b9dc916db51a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:45:54.103520 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.103505    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63fdfe2d-d11f-452a-a507-b9dc916db51a-kube-api-access-t4qwh" (OuterVolumeSpecName: "kube-api-access-t4qwh") pod "63fdfe2d-d11f-452a-a507-b9dc916db51a" (UID: "63fdfe2d-d11f-452a-a507-b9dc916db51a"). InnerVolumeSpecName "kube-api-access-t4qwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:45:54.202364 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.202338    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4qwh\" (UniqueName: \"kubernetes.io/projected/63fdfe2d-d11f-452a-a507-b9dc916db51a-kube-api-access-t4qwh\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:45:54.202364 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.202362    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63fdfe2d-d11f-452a-a507-b9dc916db51a-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:45:54.202512 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.202371    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63fdfe2d-d11f-452a-a507-b9dc916db51a-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:45:54.313389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.313310    2582 generic.go:356] "Generic (PLEG): container finished" podID="63fdfe2d-d11f-452a-a507-b9dc916db51a" containerID="11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b" exitCode=0
Jul 07 11:45:54.313389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.313354    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"
Jul 07 11:45:54.313389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.313363    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt" event={"ID":"63fdfe2d-d11f-452a-a507-b9dc916db51a","Type":"ContainerDied","Data":"11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b"}
Jul 07 11:45:54.313573 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.313410    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt" event={"ID":"63fdfe2d-d11f-452a-a507-b9dc916db51a","Type":"ContainerDied","Data":"aa0214a7bc61e73db20057b8f89b7c94e4f17c869e7d98a2ab4aee6ad8a5ffca"}
Jul 07 11:45:54.313573 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.313434    2582 scope.go:117] "RemoveContainer" containerID="11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b"
Jul 07 11:45:54.322723 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.322708    2582 scope.go:117] "RemoveContainer" containerID="11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b"
Jul 07 11:45:54.322981 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:45:54.322961    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b\": container with ID starting with 11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b not found: ID does not exist" containerID="11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b"
Jul 07 11:45:54.323040 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.322987    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b"} err="failed to get container status \"11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b\": rpc error: code = NotFound desc = could not find container \"11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b\": container with ID starting with 11dd19c4cb0865894300394bf0a6fa12b0002948d8b923ea4ebabe4db97a019b not found: ID does not exist"
Jul 07 11:45:54.343521 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.343497    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"]
Jul 07 11:45:54.348291 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:54.348274    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-23c41a78-85d69b4f7d-zkgrt"]
Jul 07 11:45:55.025349 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:55.025315    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63fdfe2d-d11f-452a-a507-b9dc916db51a" path="/var/lib/kubelet/pods/63fdfe2d-d11f-452a-a507-b9dc916db51a/volumes"
Jul 07 11:45:56.859214 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:56.859157    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:45:59.506854 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:45:59.506795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:46:04.502755 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:04.502723    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"]
Jul 07 11:46:04.605426 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:04.605390    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:46:08.832802 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:08.832773    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:46:19.522632 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.522596    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw" podUID="aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" containerName="main" containerID="cri-o://07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a" gracePeriod=45
Jul 07 11:46:19.777285 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.777226    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:46:19.815264 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815235    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-home\") pod \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") "
Jul 07 11:46:19.815382 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815293    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tls-certs\") pod \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") "
Jul 07 11:46:19.815435 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815383    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-model-cache\") pod \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") "
Jul 07 11:46:19.815435 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815422    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-dshm\") pod \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") "
Jul 07 11:46:19.815550 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815430    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-home" (OuterVolumeSpecName: "home") pod "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" (UID: "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:46:19.815550 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815447    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2c8f\" (UniqueName: \"kubernetes.io/projected/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-kube-api-access-l2c8f\") pod \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") "
Jul 07 11:46:19.815550 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815490    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tmp-dir\") pod \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\" (UID: \"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb\") "
Jul 07 11:46:19.815720 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815692    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-model-cache" (OuterVolumeSpecName: "model-cache") pod "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" (UID: "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:46:19.815828 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815808    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-home\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:46:19.815891 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815834    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-model-cache\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:46:19.815891 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.815864    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" (UID: "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:46:19.818013 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.817980    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-dshm" (OuterVolumeSpecName: "dshm") pod "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" (UID: "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:46:19.818123 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.818021    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" (UID: "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:46:19.818123 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.818029    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-kube-api-access-l2c8f" (OuterVolumeSpecName: "kube-api-access-l2c8f") pod "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" (UID: "aeeea2c9-53a9-46f1-a321-2d3dca6b19eb"). InnerVolumeSpecName "kube-api-access-l2c8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:46:19.916781 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.916750    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-dshm\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:46:19.916781 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.916777    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2c8f\" (UniqueName: \"kubernetes.io/projected/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-kube-api-access-l2c8f\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:46:19.916969 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.916788    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tmp-dir\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:46:19.916969 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:19.916797    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb-tls-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:46:20.447490 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:20.447453    2582 generic.go:356] "Generic (PLEG): container finished" podID="aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" containerID="07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a" exitCode=0
Jul 07 11:46:20.447655 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:20.447501    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"
Jul 07 11:46:20.447655 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:20.447499    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw" event={"ID":"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb","Type":"ContainerDied","Data":"07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a"}
Jul 07 11:46:20.447655 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:20.447617    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw" event={"ID":"aeeea2c9-53a9-46f1-a321-2d3dca6b19eb","Type":"ContainerDied","Data":"eeac667a5013b4c7af238401e1b66935eb2beb6fa3a9e3c1fe678941cbacb7bb"}
Jul 07 11:46:20.447655 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:20.447642    2582 scope.go:117] "RemoveContainer" containerID="07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a"
Jul 07 11:46:20.456968 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:20.456951    2582 scope.go:117] "RemoveContainer" containerID="07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a"
Jul 07 11:46:20.457232 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:46:20.457183    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a\": container with ID starting with 07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a not found: ID does not exist" containerID="07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a"
Jul 07 11:46:20.457318 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:20.457244    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a"} err="failed to get container status \"07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a\": rpc error: code = NotFound desc = could not find container \"07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a\": container with ID starting with 07ece60d5cd28f366cd310f46ef49d6d307b2b374af50aeb002088d6203cbd7a not found: ID does not exist"
Jul 07 11:46:20.471322 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:20.471288    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"]
Jul 07 11:46:20.476631 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:20.476608    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-23c41a78/e2e-lifecycle-model-23c41a78-kserve-dbbf545bc-6wlkw"]
Jul 07 11:46:21.024769 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:21.024731    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" path="/var/lib/kubelet/pods/aeeea2c9-53a9-46f1-a321-2d3dca6b19eb/volumes"
Jul 07 11:46:34.249214 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:34.249157    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:46:36.895919 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:36.895879    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:46:41.448047 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:41.447966    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:46:44.084108 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:44.084065    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:46:59.004392 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:59.004353    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:46:59.105848 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:59.105808    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:46:59.344980 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:59.344900    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"]
Jul 07 11:46:59.345130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:59.345105    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy" containerID="cri-o://08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c" gracePeriod=30
Jul 07 11:46:59.487036 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:59.486999    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:46:59.583532 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:46:59.583493    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:00.746707 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:00.746676    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:00.852597 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:00.852568    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:01.805038 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:01.805001    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:01.952137 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:01.952066    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:03.408257 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:03.408211    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:04.160336 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.160298    2582 patch_prober.go:28] interesting pod/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 11:47:04.160503 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.160367    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 11:47:04.593323 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.593298    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:47:04.613425 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613396    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pblbh\" (UniqueName: \"kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-kube-api-access-pblbh\") pod \"fa609900-0ee5-4ce3-858e-885cd97c3e58\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") "
Jul 07 11:47:04.613561 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613427    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-token\") pod \"fa609900-0ee5-4ce3-858e-885cd97c3e58\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") "
Jul 07 11:47:04.613561 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613468    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa609900-0ee5-4ce3-858e-885cd97c3e58-istiod-ca-cert\") pod \"fa609900-0ee5-4ce3-858e-885cd97c3e58\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") "
Jul 07 11:47:04.613561 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613505    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-certs\") pod \"fa609900-0ee5-4ce3-858e-885cd97c3e58\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") "
Jul 07 11:47:04.613561 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613540    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-envoy\") pod \"fa609900-0ee5-4ce3-858e-885cd97c3e58\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") "
Jul 07 11:47:04.613781 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613567    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-credential-socket\") pod \"fa609900-0ee5-4ce3-858e-885cd97c3e58\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") "
Jul 07 11:47:04.613781 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613624    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-podinfo\") pod \"fa609900-0ee5-4ce3-858e-885cd97c3e58\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") "
Jul 07 11:47:04.613781 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613657    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-socket\") pod \"fa609900-0ee5-4ce3-858e-885cd97c3e58\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") "
Jul 07 11:47:04.613781 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613701    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-data\") pod \"fa609900-0ee5-4ce3-858e-885cd97c3e58\" (UID: \"fa609900-0ee5-4ce3-858e-885cd97c3e58\") "
Jul 07 11:47:04.613981 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613868    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa609900-0ee5-4ce3-858e-885cd97c3e58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fa609900-0ee5-4ce3-858e-885cd97c3e58" (UID: "fa609900-0ee5-4ce3-858e-885cd97c3e58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:47:04.613981 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.613892    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fa609900-0ee5-4ce3-858e-885cd97c3e58" (UID: "fa609900-0ee5-4ce3-858e-885cd97c3e58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:47:04.614150 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.614126    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fa609900-0ee5-4ce3-858e-885cd97c3e58" (UID: "fa609900-0ee5-4ce3-858e-885cd97c3e58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:47:04.614239 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.614154    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fa609900-0ee5-4ce3-858e-885cd97c3e58" (UID: "fa609900-0ee5-4ce3-858e-885cd97c3e58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:47:04.614239 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.614188    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa609900-0ee5-4ce3-858e-885cd97c3e58-istiod-ca-cert\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:47:04.614239 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.614221    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-credential-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:47:04.614671 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.614651    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-data" (OuterVolumeSpecName: "istio-data") pod "fa609900-0ee5-4ce3-858e-885cd97c3e58" (UID: "fa609900-0ee5-4ce3-858e-885cd97c3e58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:47:04.615689 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.615665    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-kube-api-access-pblbh" (OuterVolumeSpecName: "kube-api-access-pblbh") pod "fa609900-0ee5-4ce3-858e-885cd97c3e58" (UID: "fa609900-0ee5-4ce3-858e-885cd97c3e58"). InnerVolumeSpecName "kube-api-access-pblbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:47:04.615806 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.615701    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fa609900-0ee5-4ce3-858e-885cd97c3e58" (UID: "fa609900-0ee5-4ce3-858e-885cd97c3e58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:47:04.616043 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.616023    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa609900-0ee5-4ce3-858e-885cd97c3e58" (UID: "fa609900-0ee5-4ce3-858e-885cd97c3e58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:47:04.616097 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.616032    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fa609900-0ee5-4ce3-858e-885cd97c3e58" (UID: "fa609900-0ee5-4ce3-858e-885cd97c3e58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:47:04.650476 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.650453    2582 generic.go:356] "Generic (PLEG): container finished" podID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerID="08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c" exitCode=0
Jul 07 11:47:04.650579 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.650507    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"
Jul 07 11:47:04.650579 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.650542    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" event={"ID":"fa609900-0ee5-4ce3-858e-885cd97c3e58","Type":"ContainerDied","Data":"08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c"}
Jul 07 11:47:04.650654 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.650597    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx" event={"ID":"fa609900-0ee5-4ce3-858e-885cd97c3e58","Type":"ContainerDied","Data":"0aa2b193a235fa50a069a8ba96c8ab5804670fc17166fd5c800d7bc17e13527f"}
Jul 07 11:47:04.650654 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.650627    2582 scope.go:117] "RemoveContainer" containerID="08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c"
Jul 07 11:47:04.663108 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.661797    2582 scope.go:117] "RemoveContainer" containerID="a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e"
Jul 07 11:47:04.672318 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.672293    2582 scope.go:117] "RemoveContainer" containerID="08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c"
Jul 07 11:47:04.672583 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:47:04.672563    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c\": container with ID starting with 08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c not found: ID does not exist" containerID="08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c"
Jul 07 11:47:04.672649 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.672590    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c"} err="failed to get container status \"08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c\": rpc error: code = NotFound desc = could not find container \"08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c\": container with ID starting with 08e92b8856886e9ff11db05426abde599b87e5190123223738744a039d4f947c not found: ID does not exist"
Jul 07 11:47:04.672649 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.672609    2582 scope.go:117] "RemoveContainer" containerID="a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e"
Jul 07 11:47:04.672862 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:47:04.672846    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e\": container with ID starting with a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e not found: ID does not exist" containerID="a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e"
Jul 07 11:47:04.672908 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.672867    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e"} err="failed to get container status \"a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e\": rpc error: code = NotFound desc = could not find container \"a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e\": container with ID starting with a6174f56bb9e9519ff59050336c5113669a5282a38b08891458bf35b3731070e not found: ID does not exist"
Jul 07 11:47:04.714752 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.714733    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-envoy\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:47:04.714752 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.714753    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-podinfo\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:47:04.714872 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.714762    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:47:04.714872 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.714771    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-data\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:47:04.714872 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.714780    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pblbh\" (UniqueName: \"kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-kube-api-access-pblbh\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:47:04.714872 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.714789    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa609900-0ee5-4ce3-858e-885cd97c3e58-istio-token\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:47:04.714872 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.714797    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa609900-0ee5-4ce3-858e-885cd97c3e58-workload-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:47:04.771142 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.771111    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"]
Jul 07 11:47:04.779137 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:04.779116    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-23c41a78-openshift-default-86c5b4cbf5-hmlhx"]
Jul 07 11:47:05.024725 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:05.024646    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" path="/var/lib/kubelet/pods/fa609900-0ee5-4ce3-858e-885cd97c3e58/volumes"
Jul 07 11:47:11.862283 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:11.862245    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:12.116153 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:12.116063    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:47:12.153004 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:12.152968    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:12.684373 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:12.684314    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:13.540704 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:13.540668    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:13.797497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:13.797403    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:19.630303 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:19.630263    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:19.835629 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:19.835599    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:19.911408 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:19.911323    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:47:19.978097 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:19.978064    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:30.424037 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:30.423989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:30.758109 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:30.758071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:51.752692 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:51.752651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:57.630237 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:57.630173    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:57.702209 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:57.702165    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:58.050103 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.050062    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:58.397278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397183    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"]
Jul 07 11:47:58.397562 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397549    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63fdfe2d-d11f-452a-a507-b9dc916db51a" containerName="maas-api"
Jul 07 11:47:58.397603 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397563    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="63fdfe2d-d11f-452a-a507-b9dc916db51a" containerName="maas-api"
Jul 07 11:47:58.397603 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397572    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy"
Jul 07 11:47:58.397603 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397577    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy"
Jul 07 11:47:58.397603 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397584    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" containerName="main"
Jul 07 11:47:58.397603 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397590    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" containerName="main"
Jul 07 11:47:58.397603 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397597    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy"
Jul 07 11:47:58.397603 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397602    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy"
Jul 07 11:47:58.397813 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397656    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeeea2c9-53a9-46f1-a321-2d3dca6b19eb" containerName="main"
Jul 07 11:47:58.397813 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397663    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy"
Jul 07 11:47:58.397813 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397669    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="63fdfe2d-d11f-452a-a507-b9dc916db51a" containerName="maas-api"
Jul 07 11:47:58.397813 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.397677    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa609900-0ee5-4ce3-858e-885cd97c3e58" containerName="istio-proxy"
Jul 07 11:47:58.400606 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.400591    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.417410 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.417385    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-a7c053-openshift-default-dockercfg-lzrnf\""
Jul 07 11:47:58.434403 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.434370    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 11:47:58.462976 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.462945    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"]
Jul 07 11:47:58.467829 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.467801    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:58.469865 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.469840    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chfns\" (UniqueName: \"kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-kube-api-access-chfns\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.469987 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.469883    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-podinfo\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.469987 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.469916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-certs\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.469987 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.469949    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-socket\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.469987 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.469970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-token\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.470155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.469990    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istiod-ca-cert\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.470155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.470012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-credential-socket\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.470155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.470034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-envoy\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.470155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.470060    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-data\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571146 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571114    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chfns\" (UniqueName: \"kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-kube-api-access-chfns\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571186    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-podinfo\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571244    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-certs\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571273    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-socket\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571300    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-token\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571344    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istiod-ca-cert\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571651 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571370    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-credential-socket\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571755 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-envoy\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571755 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571737    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-socket\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571891 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571764    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-certs\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.571891 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571800    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-data\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.572009 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.571992    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-credential-socket\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.572177 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.572151    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istiod-ca-cert\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.572293 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.572159    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-data\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.574314 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.574287    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-envoy\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.574445 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.574346    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-podinfo\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.580157 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.580137    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chfns\" (UniqueName: \"kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-kube-api-access-chfns\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.580418 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.580397    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-token\") pod \"e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") " pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.711190 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.711096    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:47:58.860024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.859994    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"]
Jul 07 11:47:58.860567 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:47:58.860543    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf1391e_e460_42e7_bdab_9d8c44cb9899.slice/crio-0c26fd2101a899b43c670e9e988d1bd4324ceb0ec7081cabce126b6e8ce03909 WatchSource:0}: Error finding container 0c26fd2101a899b43c670e9e988d1bd4324ceb0ec7081cabce126b6e8ce03909: Status 404 returned error can't find the container with id 0c26fd2101a899b43c670e9e988d1bd4324ceb0ec7081cabce126b6e8ce03909
Jul 07 11:47:58.862806 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.862769    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:47:58.862899 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.862855    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:47:58.862899 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.862890    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:47:58.900040 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.900016    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" event={"ID":"dbf1391e-e460-42e7-bdab-9d8c44cb9899","Type":"ContainerStarted","Data":"0c26fd2101a899b43c670e9e988d1bd4324ceb0ec7081cabce126b6e8ce03909"}
Jul 07 11:47:58.949095 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:58.949071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:47:59.905922 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:59.905891    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" event={"ID":"dbf1391e-e460-42e7-bdab-9d8c44cb9899","Type":"ContainerStarted","Data":"6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b"}
Jul 07 11:47:59.968447 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:47:59.968394    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" podStartSLOduration=1.968373815 podStartE2EDuration="1.968373815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:47:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:47:59.965785337 +0000 UTC m=+1961.504476761" watchObservedRunningTime="2026-07-07 11:47:59.968373815 +0000 UTC m=+1961.507065228"
Jul 07 11:48:00.711541 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:00.711503    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:48:00.716332 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:00.716307    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:48:00.910624 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:00.910591    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:48:00.911825 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:00.911800    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:48:02.155544 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.155497    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:02.230927 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.230895    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"]
Jul 07 11:48:02.234843 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.234825    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:02.239133 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.239112    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-a7c053-serving-cert\""
Jul 07 11:48:02.239273 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.239231    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-fpmsd\""
Jul 07 11:48:02.261629 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.261607    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"]
Jul 07 11:48:02.305815 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.305784    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df57fda1-9d80-4d58-b039-f93e8d073cc4-maas-api-tls\") pod \"maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") " pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:02.306002 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.305822    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df57fda1-9d80-4d58-b039-f93e8d073cc4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") " pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:02.306002 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.305843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffdcz\" (UniqueName: \"kubernetes.io/projected/df57fda1-9d80-4d58-b039-f93e8d073cc4-kube-api-access-ffdcz\") pod \"maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") " pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:02.317214 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.317166    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:02.406496 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.406398    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df57fda1-9d80-4d58-b039-f93e8d073cc4-maas-api-tls\") pod \"maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") " pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:02.406496 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.406436    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df57fda1-9d80-4d58-b039-f93e8d073cc4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") " pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:02.406496 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.406460    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffdcz\" (UniqueName: \"kubernetes.io/projected/df57fda1-9d80-4d58-b039-f93e8d073cc4-kube-api-access-ffdcz\") pod \"maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") " pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:02.406780 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:48:02.406571    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-a7c053-serving-cert: secret "maas-api-e2e-inf-a-a7c053-serving-cert" not found
Jul 07 11:48:02.406780 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:48:02.406642    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df57fda1-9d80-4d58-b039-f93e8d073cc4-maas-api-tls podName:df57fda1-9d80-4d58-b039-f93e8d073cc4 nodeName:}" failed. No retries permitted until 2026-07-07 11:48:02.906623174 +0000 UTC m=+1964.445314564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/df57fda1-9d80-4d58-b039-f93e8d073cc4-maas-api-tls") pod "maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k" (UID: "df57fda1-9d80-4d58-b039-f93e8d073cc4") : secret "maas-api-e2e-inf-a-a7c053-serving-cert" not found
Jul 07 11:48:02.407087 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.407065    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df57fda1-9d80-4d58-b039-f93e8d073cc4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") " pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:02.430298 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.430265    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffdcz\" (UniqueName: \"kubernetes.io/projected/df57fda1-9d80-4d58-b039-f93e8d073cc4-kube-api-access-ffdcz\") pod \"maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") " pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:02.628561 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.628524    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:02.911300 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.911268    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df57fda1-9d80-4d58-b039-f93e8d073cc4-maas-api-tls\") pod \"maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") " pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:02.913684 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:02.913661    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df57fda1-9d80-4d58-b039-f93e8d073cc4-maas-api-tls\") pod \"maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") " pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:03.145859 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:03.145821    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:03.254654 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:03.254624    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:03.283285 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:03.283226    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"]
Jul 07 11:48:03.285974 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:48:03.285950    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf57fda1_9d80_4d58_b039_f93e8d073cc4.slice/crio-95aec8a8027bed2c58d22e518aaa948833035038fba7b00ec0ff6c3186efff4c WatchSource:0}: Error finding container 95aec8a8027bed2c58d22e518aaa948833035038fba7b00ec0ff6c3186efff4c: Status 404 returned error can't find the container with id 95aec8a8027bed2c58d22e518aaa948833035038fba7b00ec0ff6c3186efff4c
Jul 07 11:48:03.287382 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:03.287366    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:48:03.357139 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:03.357102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:03.926263 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:03.926223    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k" event={"ID":"df57fda1-9d80-4d58-b039-f93e8d073cc4","Type":"ContainerStarted","Data":"95aec8a8027bed2c58d22e518aaa948833035038fba7b00ec0ff6c3186efff4c"}
Jul 07 11:48:04.932387 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:04.932354    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k" event={"ID":"df57fda1-9d80-4d58-b039-f93e8d073cc4","Type":"ContainerStarted","Data":"ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779"}
Jul 07 11:48:04.932763 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:04.932479    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:04.960849 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:04.960800    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k" podStartSLOduration=2.298197441 podStartE2EDuration="2.96078684s" totalImagesPullingTime="662.589399ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:48:04.958777226 +0000 UTC m=+1966.497468666" watchObservedRunningTime="2026-07-07 11:48:04.96078684 +0000 UTC m=+1966.499478251"
Jul 07 11:48:06.575661 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:06.575627    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:06.899443 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:06.899342    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:07.193098 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.193022    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"]
Jul 07 11:48:07.197036 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.197018    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.207658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.207635    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c903aa-openshift-default-dockercfg-q7g6l\""
Jul 07 11:48:07.226769 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.226745    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"]
Jul 07 11:48:07.238183 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.238150    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:07.249104 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.249080    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-socket\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.249217 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.249127    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-data\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.249278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.249211    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-podinfo\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.249278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.249237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-certs\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.249382 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.249283    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-envoy\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.249382 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.249345    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-credential-socket\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.249490 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.249395    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l72hc\" (UniqueName: \"kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-kube-api-access-l72hc\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.249490 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.249432    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-token\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.249490 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.249461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/092afb33-c2e8-494f-bb86-10facb71d7cb-istiod-ca-cert\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.350679 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.350653    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-podinfo\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.350870 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.350690    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-certs\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.350870 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.350749    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-envoy\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.350870 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.350785    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-credential-socket\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.351024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.350929    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l72hc\" (UniqueName: \"kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-kube-api-access-l72hc\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.351024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.350986    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-token\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.351024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.351015    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/092afb33-c2e8-494f-bb86-10facb71d7cb-istiod-ca-cert\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.351167 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.351056    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-socket\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.351167 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.351101    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-data\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.351326 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.351172    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-certs\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.351627 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.351596    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-credential-socket\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.351747 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.351603    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-socket\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.351747 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.351726    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-data\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.351747 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.351742    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/092afb33-c2e8-494f-bb86-10facb71d7cb-istiod-ca-cert\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.353120 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.353099    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-envoy\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.353277 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.353261    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-podinfo\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.365583 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.365555    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-token\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.399230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.399175    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l72hc\" (UniqueName: \"kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-kube-api-access-l72hc\") pod \"e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") " pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.467998 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.467957    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:07.506846 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.506810    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:07.634705 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.634672    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:48:07.650297 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.650270    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"]
Jul 07 11:48:07.653051 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:48:07.653024    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod092afb33_c2e8_494f_bb86_10facb71d7cb.slice/crio-c10f1cc6f2cc9d4724b73fab00ca5a7f5fc5d1570d15ca8c22fd65897eb5d26d WatchSource:0}: Error finding container c10f1cc6f2cc9d4724b73fab00ca5a7f5fc5d1570d15ca8c22fd65897eb5d26d: Status 404 returned error can't find the container with id c10f1cc6f2cc9d4724b73fab00ca5a7f5fc5d1570d15ca8c22fd65897eb5d26d
Jul 07 11:48:07.655171 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.655135    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:48:07.655254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.655241    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:48:07.655292 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.655281    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:48:07.947773 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.947688    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" event={"ID":"092afb33-c2e8-494f-bb86-10facb71d7cb","Type":"ContainerStarted","Data":"d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc"}
Jul 07 11:48:07.947773 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.947743    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" event={"ID":"092afb33-c2e8-494f-bb86-10facb71d7cb","Type":"ContainerStarted","Data":"c10f1cc6f2cc9d4724b73fab00ca5a7f5fc5d1570d15ca8c22fd65897eb5d26d"}
Jul 07 11:48:07.973040 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:07.972995    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" podStartSLOduration=0.972974052 podStartE2EDuration="972.974052ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:48:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:48:07.971530958 +0000 UTC m=+1969.510222381" watchObservedRunningTime="2026-07-07 11:48:07.972974052 +0000 UTC m=+1969.511665502"
Jul 07 11:48:08.507723 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:08.507687    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:08.512645 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:08.512622    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:08.952706 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:08.952677    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:08.953829 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:08.953807    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:10.180535 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.180402    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:10.197684 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.187554    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"]
Jul 07 11:48:10.197684 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.195762    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:10.201239 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.201188    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-c903aa-serving-cert\""
Jul 07 11:48:10.202824 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.202796    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"]
Jul 07 11:48:10.284799 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.284771    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:10.287633 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.287600    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:10.287776 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.287723    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30dca09-3318-489b-97be-a96330f6c24c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:10.287776 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.287767    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfrsz\" (UniqueName: \"kubernetes.io/projected/d30dca09-3318-489b-97be-a96330f6c24c-kube-api-access-tfrsz\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:10.389317 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.389267    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfrsz\" (UniqueName: \"kubernetes.io/projected/d30dca09-3318-489b-97be-a96330f6c24c-kube-api-access-tfrsz\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:10.390343 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:48:10.390045    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-c903aa-serving-cert: secret "maas-api-e2e-inf-b-c903aa-serving-cert" not found
Jul 07 11:48:10.390343 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:48:10.390119    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls podName:d30dca09-3318-489b-97be-a96330f6c24c nodeName:}" failed. No retries permitted until 2026-07-07 11:48:10.89009779 +0000 UTC m=+1972.428789182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls") pod "maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt" (UID: "d30dca09-3318-489b-97be-a96330f6c24c") : secret "maas-api-e2e-inf-b-c903aa-serving-cert" not found
Jul 07 11:48:10.390806 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.390595    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:10.390806 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.390738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30dca09-3318-489b-97be-a96330f6c24c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:10.391622 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.391593    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30dca09-3318-489b-97be-a96330f6c24c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:10.402264 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.402241    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfrsz\" (UniqueName: \"kubernetes.io/projected/d30dca09-3318-489b-97be-a96330f6c24c-kube-api-access-tfrsz\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:10.697533 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.697497    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:10.800868 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.800830    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:10.895567 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.895522    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:10.895748 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:48:10.895679    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-c903aa-serving-cert: secret "maas-api-e2e-inf-b-c903aa-serving-cert" not found
Jul 07 11:48:10.895788 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:48:10.895752    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls podName:d30dca09-3318-489b-97be-a96330f6c24c nodeName:}" failed. No retries permitted until 2026-07-07 11:48:11.895733136 +0000 UTC m=+1973.434424530 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls") pod "maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt" (UID: "d30dca09-3318-489b-97be-a96330f6c24c") : secret "maas-api-e2e-inf-b-c903aa-serving-cert" not found
Jul 07 11:48:10.942067 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:10.942039    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:48:11.161157 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:11.161122    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:11.906306 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:11.906264    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:11.908701 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:11.908675    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls\") pod \"maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") " pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:12.012560 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:12.012524    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:12.188641 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:12.188484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"]
Jul 07 11:48:12.193257 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:48:12.193225    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd30dca09_3318_489b_97be_a96330f6c24c.slice/crio-12ad1b373ccbe2a4a2494c5d00b015a2207c5f1f41822bbd69dccf3ca42565a5 WatchSource:0}: Error finding container 12ad1b373ccbe2a4a2494c5d00b015a2207c5f1f41822bbd69dccf3ca42565a5: Status 404 returned error can't find the container with id 12ad1b373ccbe2a4a2494c5d00b015a2207c5f1f41822bbd69dccf3ca42565a5
Jul 07 11:48:12.977357 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:12.977324    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt" event={"ID":"d30dca09-3318-489b-97be-a96330f6c24c","Type":"ContainerStarted","Data":"7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c"}
Jul 07 11:48:12.977775 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:12.977382    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt" event={"ID":"d30dca09-3318-489b-97be-a96330f6c24c","Type":"ContainerStarted","Data":"12ad1b373ccbe2a4a2494c5d00b015a2207c5f1f41822bbd69dccf3ca42565a5"}
Jul 07 11:48:12.977775 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:12.977456    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:13.001016 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:13.000958    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt" podStartSLOduration=2.51369082 podStartE2EDuration="3.000937541s" totalImagesPullingTime="487.246721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:48:12.998757325 +0000 UTC m=+1974.537448737" watchObservedRunningTime="2026-07-07 11:48:13.000937541 +0000 UTC m=+1974.539628954"
Jul 07 11:48:14.515054 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:14.515019    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:15.912140 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:15.912056    2582 patch_prober.go:28] interesting pod/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 07 11:48:15.912140 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:15.912114    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 07 11:48:15.996397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:15.996363    2582 generic.go:356] "Generic (PLEG): container finished" podID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerID="6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b" exitCode=0
Jul 07 11:48:15.996581 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:15.996434    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" event={"ID":"dbf1391e-e460-42e7-bdab-9d8c44cb9899","Type":"ContainerDied","Data":"6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b"}
Jul 07 11:48:15.996899 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:15.996883    2582 scope.go:117] "RemoveContainer" containerID="6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b"
Jul 07 11:48:15.997881 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:15.997854    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:48:15.997969 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:15.997914    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:48:15.997969 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:15.997949    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:48:17.002768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:17.002731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" event={"ID":"dbf1391e-e460-42e7-bdab-9d8c44cb9899","Type":"ContainerStarted","Data":"d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5"}
Jul 07 11:48:17.712438 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:17.712405    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:48:17.712790 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:17.712770    2582 patch_prober.go:28] interesting pod/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 07 11:48:17.712940 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:17.712913    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 07 11:48:18.712452 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:18.712414    2582 patch_prober.go:28] interesting pod/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 07 11:48:18.712967 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:18.712472    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 07 11:48:18.997392 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:18.997301    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:48:19.712404 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:19.712310    2582 patch_prober.go:28] interesting pod/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 07 11:48:19.712404 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:19.712372    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 07 11:48:20.711939 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:20.711900    2582 patch_prober.go:28] interesting pod/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 07 11:48:20.712146 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:20.711965    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 07 11:48:21.716340 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:21.716307    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:48:21.716756 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:21.716601    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:48:21.717372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:21.717354    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:48:22.887914 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:22.887875    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"]
Jul 07 11:48:22.891407 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:22.891391    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:22.894882 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:22.894862    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a7c053\"/\"openshift-service-ca.crt\""
Jul 07 11:48:22.894993 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:22.894868    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a7c053\"/\"default-dockercfg-rqb75\""
Jul 07 11:48:22.894993 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:22.894956    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a7c053\"/\"test-model-a7c053-kserve-self-signed-certs\""
Jul 07 11:48:22.895084 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:22.895064    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a7c053\"/\"kube-root-ca.crt\""
Jul 07 11:48:22.899849 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:22.899829    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"]
Jul 07 11:48:22.914059 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:22.914035    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:23.012012 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.011978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20f6020-24b8-4475-989a-fe722f793693-tls-certs\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.012222 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.012022    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-model-cache\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.012222 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.012084    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-dshm\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.012222 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.012131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md5wv\" (UniqueName: \"kubernetes.io/projected/e20f6020-24b8-4475-989a-fe722f793693-kube-api-access-md5wv\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.012222 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.012163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-tmp-dir\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.012403 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.012275    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-home\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.025623 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.025597    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:48:23.046724 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.046691    2582 generic.go:356] "Generic (PLEG): container finished" podID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerID="d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc" exitCode=0
Jul 07 11:48:23.046864 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.046762    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" event={"ID":"092afb33-c2e8-494f-bb86-10facb71d7cb","Type":"ContainerDied","Data":"d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc"}
Jul 07 11:48:23.047276 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.047254    2582 scope.go:117] "RemoveContainer" containerID="d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc"
Jul 07 11:48:23.048211 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.048169    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:48:23.048326 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.048276    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:48:23.048394 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.048330    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:48:23.113299 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.113265    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20f6020-24b8-4475-989a-fe722f793693-tls-certs\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.113407 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.113352    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-model-cache\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.113407 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.113388    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-dshm\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.113529 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.113461    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md5wv\" (UniqueName: \"kubernetes.io/projected/e20f6020-24b8-4475-989a-fe722f793693-kube-api-access-md5wv\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.113529 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.113509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-tmp-dir\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.114294 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.113718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-home\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.114294 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.113963    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-tmp-dir\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.114294 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.114047    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-home\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.114826 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.114786    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-model-cache\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.116589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.116567    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-dshm\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.116820 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.116799    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20f6020-24b8-4475-989a-fe722f793693-tls-certs\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.123172 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.123147    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md5wv\" (UniqueName: \"kubernetes.io/projected/e20f6020-24b8-4475-989a-fe722f793693-kube-api-access-md5wv\") pod \"test-model-a7c053-kserve-d97666d59-774g7\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") " pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.200997 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.200920    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:23.349349 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.349326    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"]
Jul 07 11:48:23.351706 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:48:23.351679    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode20f6020_24b8_4475_989a_fe722f793693.slice/crio-62e1c34fa268c4b21090c3ca586a2173a31de01698415c7415e1cc43423d341e WatchSource:0}: Error finding container 62e1c34fa268c4b21090c3ca586a2173a31de01698415c7415e1cc43423d341e: Status 404 returned error can't find the container with id 62e1c34fa268c4b21090c3ca586a2173a31de01698415c7415e1cc43423d341e
Jul 07 11:48:23.953445 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:23.953415    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:24.053556 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:24.053512    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" event={"ID":"092afb33-c2e8-494f-bb86-10facb71d7cb","Type":"ContainerStarted","Data":"9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425"}
Jul 07 11:48:24.054976 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:24.054949    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7" event={"ID":"e20f6020-24b8-4475-989a-fe722f793693","Type":"ContainerStarted","Data":"3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953"}
Jul 07 11:48:24.055103 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:24.054985    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7" event={"ID":"e20f6020-24b8-4475-989a-fe722f793693","Type":"ContainerStarted","Data":"62e1c34fa268c4b21090c3ca586a2173a31de01698415c7415e1cc43423d341e"}
Jul 07 11:48:24.096260 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:24.096167    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7" podStartSLOduration=2.096154017 podStartE2EDuration="2.096154017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:48:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:48:24.094674415 +0000 UTC m=+1985.633365827" watchObservedRunningTime="2026-07-07 11:48:24.096154017 +0000 UTC m=+1985.634845429"
Jul 07 11:48:24.506950 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:24.506913    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:24.508254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:24.508219    2582 patch_prober.go:28] interesting pod/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 07 11:48:24.508395 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:24.508276    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 07 11:48:24.609220 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:24.609151    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:24.715447 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:24.715410    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:25.507643 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:25.507602    2582 patch_prober.go:28] interesting pod/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 07 11:48:25.508035 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:25.507674    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 07 11:48:26.507688 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:26.507653    2582 patch_prober.go:28] interesting pod/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 07 11:48:26.508082 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:26.507715    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 07 11:48:27.507812 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:27.507773    2582 patch_prober.go:28] interesting pod/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 07 11:48:27.508257 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:27.507830    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 07 11:48:28.511077 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:28.511045    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:28.511505 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:28.511421    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:28.512113 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:28.512097    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:48:31.440354 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:31.440311    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:31.543308 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:31.543267    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:33.201485 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:33.201454    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:33.201865 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:33.201520    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:33.214550 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:33.214524    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:34.120641 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:34.120610    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:48:36.127095 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:36.127067    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:48:47.690187 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:47.690140    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:48:51.522497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:48:51.522463    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:49:04.528971 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:04.528934    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:49:12.977249 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:12.977208    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"]
Jul 07 11:49:12.996217 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:12.996161    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:14.041239 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:14.041177    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:14.201827 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:14.201792    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:14.596924 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:14.596888    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:14.691968 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:14.691934    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:14.790859 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:14.790827    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:14.970004 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:14.969973    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"]
Jul 07 11:49:14.970234 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:14.970182    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k" podUID="df57fda1-9d80-4d58-b039-f93e8d073cc4" containerName="maas-api" containerID="cri-o://ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779" gracePeriod=30
Jul 07 11:49:15.045759 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.045724    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:15.222153 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.222094    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:49:15.279764 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.279733    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df57fda1-9d80-4d58-b039-f93e8d073cc4-maas-api-tls\") pod \"df57fda1-9d80-4d58-b039-f93e8d073cc4\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") "
Jul 07 11:49:15.279895 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.279852    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffdcz\" (UniqueName: \"kubernetes.io/projected/df57fda1-9d80-4d58-b039-f93e8d073cc4-kube-api-access-ffdcz\") pod \"df57fda1-9d80-4d58-b039-f93e8d073cc4\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") "
Jul 07 11:49:15.279976 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.279899    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df57fda1-9d80-4d58-b039-f93e8d073cc4-openshift-service-ca\") pod \"df57fda1-9d80-4d58-b039-f93e8d073cc4\" (UID: \"df57fda1-9d80-4d58-b039-f93e8d073cc4\") "
Jul 07 11:49:15.280381 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.280343    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df57fda1-9d80-4d58-b039-f93e8d073cc4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "df57fda1-9d80-4d58-b039-f93e8d073cc4" (UID: "df57fda1-9d80-4d58-b039-f93e8d073cc4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:49:15.281795 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.281765    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df57fda1-9d80-4d58-b039-f93e8d073cc4-kube-api-access-ffdcz" (OuterVolumeSpecName: "kube-api-access-ffdcz") pod "df57fda1-9d80-4d58-b039-f93e8d073cc4" (UID: "df57fda1-9d80-4d58-b039-f93e8d073cc4"). InnerVolumeSpecName "kube-api-access-ffdcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:49:15.281914 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.281768    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df57fda1-9d80-4d58-b039-f93e8d073cc4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "df57fda1-9d80-4d58-b039-f93e8d073cc4" (UID: "df57fda1-9d80-4d58-b039-f93e8d073cc4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:49:15.305007 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.304981    2582 generic.go:356] "Generic (PLEG): container finished" podID="df57fda1-9d80-4d58-b039-f93e8d073cc4" containerID="ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779" exitCode=0
Jul 07 11:49:15.305100 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.305033    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"
Jul 07 11:49:15.305100 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.305066    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k" event={"ID":"df57fda1-9d80-4d58-b039-f93e8d073cc4","Type":"ContainerDied","Data":"ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779"}
Jul 07 11:49:15.305179 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.305118    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k" event={"ID":"df57fda1-9d80-4d58-b039-f93e8d073cc4","Type":"ContainerDied","Data":"95aec8a8027bed2c58d22e518aaa948833035038fba7b00ec0ff6c3186efff4c"}
Jul 07 11:49:15.305179 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.305141    2582 scope.go:117] "RemoveContainer" containerID="ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779"
Jul 07 11:49:15.314071 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.314055    2582 scope.go:117] "RemoveContainer" containerID="ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779"
Jul 07 11:49:15.314315 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:49:15.314296    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779\": container with ID starting with ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779 not found: ID does not exist" containerID="ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779"
Jul 07 11:49:15.314373 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.314324    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779"} err="failed to get container status \"ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779\": rpc error: code = NotFound desc = could not find container \"ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779\": container with ID starting with ce8371102129e0e639007eb7cdd9320e4a56dcfe5c0580e5752259b5cd8dd779 not found: ID does not exist"
Jul 07 11:49:15.331025 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.331004    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"]
Jul 07 11:49:15.335891 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.335871    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-a7c053-799756ffbb-2bv5k"]
Jul 07 11:49:15.380956 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.380929    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffdcz\" (UniqueName: \"kubernetes.io/projected/df57fda1-9d80-4d58-b039-f93e8d073cc4-kube-api-access-ffdcz\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:15.380956 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.380953    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df57fda1-9d80-4d58-b039-f93e8d073cc4-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:15.381109 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:15.380964    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df57fda1-9d80-4d58-b039-f93e8d073cc4-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:16.729185 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:16.729148    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:49:17.027305 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:17.027220    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df57fda1-9d80-4d58-b039-f93e8d073cc4" path="/var/lib/kubelet/pods/df57fda1-9d80-4d58-b039-f93e8d073cc4/volumes"
Jul 07 11:49:19.663814 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:19.663779    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:27.997721 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:27.997665    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7" podUID="e20f6020-24b8-4475-989a-fe722f793693" containerName="main" containerID="cri-o://3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953" gracePeriod=45
Jul 07 11:49:28.252753 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.252694    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:49:28.367937 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.367904    2582 generic.go:356] "Generic (PLEG): container finished" podID="e20f6020-24b8-4475-989a-fe722f793693" containerID="3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953" exitCode=0
Jul 07 11:49:28.368119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.367959    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"
Jul 07 11:49:28.368119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.367992    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7" event={"ID":"e20f6020-24b8-4475-989a-fe722f793693","Type":"ContainerDied","Data":"3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953"}
Jul 07 11:49:28.368119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.368040    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7" event={"ID":"e20f6020-24b8-4475-989a-fe722f793693","Type":"ContainerDied","Data":"62e1c34fa268c4b21090c3ca586a2173a31de01698415c7415e1cc43423d341e"}
Jul 07 11:49:28.368119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.368065    2582 scope.go:117] "RemoveContainer" containerID="3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953"
Jul 07 11:49:28.386057 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.386037    2582 scope.go:117] "RemoveContainer" containerID="3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953"
Jul 07 11:49:28.386359 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:49:28.386340    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953\": container with ID starting with 3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953 not found: ID does not exist" containerID="3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953"
Jul 07 11:49:28.386422 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.386370    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953"} err="failed to get container status \"3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953\": rpc error: code = NotFound desc = could not find container \"3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953\": container with ID starting with 3dcd1305ce4d6b385c6e9aef2cc8c237c25425e2a4be9a8f79acb112126da953 not found: ID does not exist"
Jul 07 11:49:28.398050 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398033    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20f6020-24b8-4475-989a-fe722f793693-tls-certs\") pod \"e20f6020-24b8-4475-989a-fe722f793693\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") "
Jul 07 11:49:28.398143 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398065    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md5wv\" (UniqueName: \"kubernetes.io/projected/e20f6020-24b8-4475-989a-fe722f793693-kube-api-access-md5wv\") pod \"e20f6020-24b8-4475-989a-fe722f793693\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") "
Jul 07 11:49:28.398143 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398092    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-model-cache\") pod \"e20f6020-24b8-4475-989a-fe722f793693\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") "
Jul 07 11:49:28.398143 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398128    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-tmp-dir\") pod \"e20f6020-24b8-4475-989a-fe722f793693\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") "
Jul 07 11:49:28.398297 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398154    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-home\") pod \"e20f6020-24b8-4475-989a-fe722f793693\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") "
Jul 07 11:49:28.398297 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398247    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-dshm\") pod \"e20f6020-24b8-4475-989a-fe722f793693\" (UID: \"e20f6020-24b8-4475-989a-fe722f793693\") "
Jul 07 11:49:28.398447 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398422    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-model-cache" (OuterVolumeSpecName: "model-cache") pod "e20f6020-24b8-4475-989a-fe722f793693" (UID: "e20f6020-24b8-4475-989a-fe722f793693"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:49:28.398447 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398436    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e20f6020-24b8-4475-989a-fe722f793693" (UID: "e20f6020-24b8-4475-989a-fe722f793693"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:49:28.398552 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398518    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-home" (OuterVolumeSpecName: "home") pod "e20f6020-24b8-4475-989a-fe722f793693" (UID: "e20f6020-24b8-4475-989a-fe722f793693"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:49:28.398631 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398613    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-model-cache\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:28.398691 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398658    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-tmp-dir\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:28.398691 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.398668    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-home\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:28.400274 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.400237    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20f6020-24b8-4475-989a-fe722f793693-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e20f6020-24b8-4475-989a-fe722f793693" (UID: "e20f6020-24b8-4475-989a-fe722f793693"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:49:28.400372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.400275    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20f6020-24b8-4475-989a-fe722f793693-kube-api-access-md5wv" (OuterVolumeSpecName: "kube-api-access-md5wv") pod "e20f6020-24b8-4475-989a-fe722f793693" (UID: "e20f6020-24b8-4475-989a-fe722f793693"). InnerVolumeSpecName "kube-api-access-md5wv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:49:28.400372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.400362    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-dshm" (OuterVolumeSpecName: "dshm") pod "e20f6020-24b8-4475-989a-fe722f793693" (UID: "e20f6020-24b8-4475-989a-fe722f793693"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:49:28.428938 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.428912    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:49:28.499441 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.499417    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20f6020-24b8-4475-989a-fe722f793693-tls-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:28.499547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.499453    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md5wv\" (UniqueName: \"kubernetes.io/projected/e20f6020-24b8-4475-989a-fe722f793693-kube-api-access-md5wv\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:28.499547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.499480    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20f6020-24b8-4475-989a-fe722f793693-dshm\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:28.690404 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.690370    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"]
Jul 07 11:49:28.694504 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:28.694480    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a7c053/test-model-a7c053-kserve-d97666d59-774g7"]
Jul 07 11:49:29.024863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:29.024826    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e20f6020-24b8-4475-989a-fe722f793693" path="/var/lib/kubelet/pods/e20f6020-24b8-4475-989a-fe722f793693/volumes"
Jul 07 11:49:31.484226 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:31.484159    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:44.524153 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:44.524069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:44.620360 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:44.620318    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:56.424062 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:56.424023    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:56.675645 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:56.675566    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"]
Jul 07 11:49:56.675848 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:56.675824    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy" containerID="cri-o://d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5" gracePeriod=30
Jul 07 11:49:56.699068 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:56.699025    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:56.919046 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:56.919008    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:57.090320 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:57.090289    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:57.492763 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:57.492714    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:57.593443 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:57.593408    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:57.710494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:57.710458    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"]
Jul 07 11:49:57.710762 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:57.710708    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt" podUID="d30dca09-3318-489b-97be-a96330f6c24c" containerName="maas-api" containerID="cri-o://7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c" gracePeriod=30
Jul 07 11:49:57.735355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:57.735330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:49:58.965143 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:58.965118    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:49:59.052016 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.051944    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls\") pod \"d30dca09-3318-489b-97be-a96330f6c24c\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") "
Jul 07 11:49:59.052016 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.051993    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30dca09-3318-489b-97be-a96330f6c24c-openshift-service-ca\") pod \"d30dca09-3318-489b-97be-a96330f6c24c\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") "
Jul 07 11:49:59.052188 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.052090    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfrsz\" (UniqueName: \"kubernetes.io/projected/d30dca09-3318-489b-97be-a96330f6c24c-kube-api-access-tfrsz\") pod \"d30dca09-3318-489b-97be-a96330f6c24c\" (UID: \"d30dca09-3318-489b-97be-a96330f6c24c\") "
Jul 07 11:49:59.052460 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.052438    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d30dca09-3318-489b-97be-a96330f6c24c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d30dca09-3318-489b-97be-a96330f6c24c" (UID: "d30dca09-3318-489b-97be-a96330f6c24c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:49:59.054138 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.054104    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d30dca09-3318-489b-97be-a96330f6c24c" (UID: "d30dca09-3318-489b-97be-a96330f6c24c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:49:59.054138 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.054121    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d30dca09-3318-489b-97be-a96330f6c24c-kube-api-access-tfrsz" (OuterVolumeSpecName: "kube-api-access-tfrsz") pod "d30dca09-3318-489b-97be-a96330f6c24c" (UID: "d30dca09-3318-489b-97be-a96330f6c24c"). InnerVolumeSpecName "kube-api-access-tfrsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:49:59.153038 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.153007    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d30dca09-3318-489b-97be-a96330f6c24c-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:59.153038 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.153033    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30dca09-3318-489b-97be-a96330f6c24c-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:59.153038 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.153042    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfrsz\" (UniqueName: \"kubernetes.io/projected/d30dca09-3318-489b-97be-a96330f6c24c-kube-api-access-tfrsz\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:49:59.513383 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.513347    2582 generic.go:356] "Generic (PLEG): container finished" podID="d30dca09-3318-489b-97be-a96330f6c24c" containerID="7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c" exitCode=0
Jul 07 11:49:59.513547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.513400    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"
Jul 07 11:49:59.513547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.513424    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt" event={"ID":"d30dca09-3318-489b-97be-a96330f6c24c","Type":"ContainerDied","Data":"7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c"}
Jul 07 11:49:59.513547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.513501    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt" event={"ID":"d30dca09-3318-489b-97be-a96330f6c24c","Type":"ContainerDied","Data":"12ad1b373ccbe2a4a2494c5d00b015a2207c5f1f41822bbd69dccf3ca42565a5"}
Jul 07 11:49:59.513547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.513523    2582 scope.go:117] "RemoveContainer" containerID="7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c"
Jul 07 11:49:59.522581 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.522561    2582 scope.go:117] "RemoveContainer" containerID="7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c"
Jul 07 11:49:59.522827 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:49:59.522811    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c\": container with ID starting with 7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c not found: ID does not exist" containerID="7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c"
Jul 07 11:49:59.522873 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.522835    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c"} err="failed to get container status \"7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c\": rpc error: code = NotFound desc = could not find container \"7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c\": container with ID starting with 7a16dd20e4e1e2e21edc83fa604438ff22247530fc8ef1fb8059af050e8db14c not found: ID does not exist"
Jul 07 11:49:59.534289 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.534261    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"]
Jul 07 11:49:59.539291 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.539271    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c903aa-6f4d5d95f-mmsrt"]
Jul 07 11:49:59.621888 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:49:59.621855    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:50:01.025363 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.025329    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d30dca09-3318-489b-97be-a96330f6c24c" path="/var/lib/kubelet/pods/d30dca09-3318-489b-97be-a96330f6c24c/volumes"
Jul 07 11:50:01.218888 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.218855    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:01.292340 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.292236    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:01.924696 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.924675    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:50:01.976436 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976414    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-podinfo\") pod \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") "
Jul 07 11:50:01.976558 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976457    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-socket\") pod \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") "
Jul 07 11:50:01.976558 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976491    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-credential-socket\") pod \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") "
Jul 07 11:50:01.976558 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976517    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istiod-ca-cert\") pod \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") "
Jul 07 11:50:01.976721 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976642    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chfns\" (UniqueName: \"kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-kube-api-access-chfns\") pod \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") "
Jul 07 11:50:01.976721 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976684    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-data\") pod \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") "
Jul 07 11:50:01.976841 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976720    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-envoy\") pod \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") "
Jul 07 11:50:01.976841 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976751    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dbf1391e-e460-42e7-bdab-9d8c44cb9899" (UID: "dbf1391e-e460-42e7-bdab-9d8c44cb9899"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:01.976841 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976774    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-certs\") pod \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") "
Jul 07 11:50:01.976841 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976821    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-token\") pod \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\" (UID: \"dbf1391e-e460-42e7-bdab-9d8c44cb9899\") "
Jul 07 11:50:01.976841 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976812    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dbf1391e-e460-42e7-bdab-9d8c44cb9899" (UID: "dbf1391e-e460-42e7-bdab-9d8c44cb9899"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:01.977118 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976953    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dbf1391e-e460-42e7-bdab-9d8c44cb9899" (UID: "dbf1391e-e460-42e7-bdab-9d8c44cb9899"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:01.977118 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.976920    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dbf1391e-e460-42e7-bdab-9d8c44cb9899" (UID: "dbf1391e-e460-42e7-bdab-9d8c44cb9899"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:50:01.977254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.977126    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istiod-ca-cert\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:01.977254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.977144    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:01.977254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.977157    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-workload-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:01.977254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.977173    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-credential-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:01.977558 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.977520    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-data" (OuterVolumeSpecName: "istio-data") pod "dbf1391e-e460-42e7-bdab-9d8c44cb9899" (UID: "dbf1391e-e460-42e7-bdab-9d8c44cb9899"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:01.978848 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.978824    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dbf1391e-e460-42e7-bdab-9d8c44cb9899" (UID: "dbf1391e-e460-42e7-bdab-9d8c44cb9899"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:01.979018 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.978999    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dbf1391e-e460-42e7-bdab-9d8c44cb9899" (UID: "dbf1391e-e460-42e7-bdab-9d8c44cb9899"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:50:01.979278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.979261    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-kube-api-access-chfns" (OuterVolumeSpecName: "kube-api-access-chfns") pod "dbf1391e-e460-42e7-bdab-9d8c44cb9899" (UID: "dbf1391e-e460-42e7-bdab-9d8c44cb9899"). InnerVolumeSpecName "kube-api-access-chfns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:01.979358 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:01.979340    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-token" (OuterVolumeSpecName: "istio-token") pod "dbf1391e-e460-42e7-bdab-9d8c44cb9899" (UID: "dbf1391e-e460-42e7-bdab-9d8c44cb9899"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:02.078511 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.078487    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-podinfo\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:02.078511 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.078510    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chfns\" (UniqueName: \"kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-kube-api-access-chfns\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:02.078843 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.078519    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-data\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:02.078843 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.078527    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-envoy\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:02.078843 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.078534    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbf1391e-e460-42e7-bdab-9d8c44cb9899-istio-token\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:02.529911 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.529877    2582 generic.go:356] "Generic (PLEG): container finished" podID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerID="d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5" exitCode=0
Jul 07 11:50:02.530094 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.529914    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" event={"ID":"dbf1391e-e460-42e7-bdab-9d8c44cb9899","Type":"ContainerDied","Data":"d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5"}
Jul 07 11:50:02.530094 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.529933    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"
Jul 07 11:50:02.530094 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.529983    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt" event={"ID":"dbf1391e-e460-42e7-bdab-9d8c44cb9899","Type":"ContainerDied","Data":"0c26fd2101a899b43c670e9e988d1bd4324ceb0ec7081cabce126b6e8ce03909"}
Jul 07 11:50:02.530094 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.530015    2582 scope.go:117] "RemoveContainer" containerID="d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5"
Jul 07 11:50:02.540318 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.540289    2582 scope.go:117] "RemoveContainer" containerID="6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b"
Jul 07 11:50:02.549078 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.549060    2582 scope.go:117] "RemoveContainer" containerID="d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5"
Jul 07 11:50:02.549380 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:02.549362    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5\": container with ID starting with d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5 not found: ID does not exist" containerID="d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5"
Jul 07 11:50:02.549437 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.549390    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5"} err="failed to get container status \"d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5\": rpc error: code = NotFound desc = could not find container \"d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5\": container with ID starting with d09cd23dcfeea62a8b59695a2a62e778a8b75128bfd04ee3ea1f121339a1a9a5 not found: ID does not exist"
Jul 07 11:50:02.549437 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.549406    2582 scope.go:117] "RemoveContainer" containerID="6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b"
Jul 07 11:50:02.549642 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:02.549627    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b\": container with ID starting with 6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b not found: ID does not exist" containerID="6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b"
Jul 07 11:50:02.549684 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.549647    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b"} err="failed to get container status \"6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b\": rpc error: code = NotFound desc = could not find container \"6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b\": container with ID starting with 6e0434a23e2b0fd78aff22ffb1051bda4b24f09ad2c2dfb00160125effde147b not found: ID does not exist"
Jul 07 11:50:02.553742 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.553720    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"]
Jul 07 11:50:02.557295 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:02.557266    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-a7c053-openshift-default-6c86d8476c-gzhgt"]
Jul 07 11:50:03.025433 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:03.025401    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" path="/var/lib/kubelet/pods/dbf1391e-e460-42e7-bdab-9d8c44cb9899/volumes"
Jul 07 11:50:03.535946 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:03.535903    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:03.636567 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:03.636532    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:03.782118 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:03.782081    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"]
Jul 07 11:50:03.782423 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:03.782400    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy" containerID="cri-o://9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425" gracePeriod=30
Jul 07 11:50:03.810575 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:03.810485    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:04.026927 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.026894    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:04.640762 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.640720    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:04.681063 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681034    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"]
Jul 07 11:50:04.681434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681417    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy"
Jul 07 11:50:04.681434 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681433    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681446    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d30dca09-3318-489b-97be-a96330f6c24c" containerName="maas-api"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681452    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d30dca09-3318-489b-97be-a96330f6c24c" containerName="maas-api"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681472    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20f6020-24b8-4475-989a-fe722f793693" containerName="main"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681480    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20f6020-24b8-4475-989a-fe722f793693" containerName="main"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681486    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681491    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681502    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df57fda1-9d80-4d58-b039-f93e8d073cc4" containerName="maas-api"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681508    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="df57fda1-9d80-4d58-b039-f93e8d073cc4" containerName="maas-api"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681574    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d30dca09-3318-489b-97be-a96330f6c24c" containerName="maas-api"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681584    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="df57fda1-9d80-4d58-b039-f93e8d073cc4" containerName="maas-api"
Jul 07 11:50:04.681589 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681595    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy"
Jul 07 11:50:04.681942 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681600    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbf1391e-e460-42e7-bdab-9d8c44cb9899" containerName="istio-proxy"
Jul 07 11:50:04.681942 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.681606    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20f6020-24b8-4475-989a-fe722f793693" containerName="main"
Jul 07 11:50:04.686139 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.686121    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.689262 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.689239    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-daa276-openshift-default-dockercfg-vj4xk\""
Jul 07 11:50:04.698163 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.698140    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"]
Jul 07 11:50:04.723114 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.723090    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:04.802796 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.802771    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abadecad-01d5-4e69-8856-7c1a2d7ee848-istiod-ca-cert\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.802911 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.802803    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-socket\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.802911 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.802826    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-certs\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.803004 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.802939    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-envoy\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.803004 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.802964    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-data\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.803004 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.802985    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-credential-socket\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.803144 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.803007    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpsx7\" (UniqueName: \"kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-kube-api-access-hpsx7\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.803144 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.803053    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-podinfo\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.803144 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.803123    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-token\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.889911 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.889882    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:04.904165 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904103    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abadecad-01d5-4e69-8856-7c1a2d7ee848-istiod-ca-cert\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904165 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904133    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-socket\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904165 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904155    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-certs\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904226    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-envoy\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904248    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-data\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904275    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-credential-socket\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpsx7\" (UniqueName: \"kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-kube-api-access-hpsx7\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904324    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-podinfo\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-token\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904736 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904591    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-socket\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904736 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904637    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-data\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904855 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904801    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-certs\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.904913 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.904894    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-credential-socket\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.905058 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.905039    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abadecad-01d5-4e69-8856-7c1a2d7ee848-istiod-ca-cert\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.906527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.906498    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-envoy\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.906798 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.906777    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-podinfo\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.917173 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.917156    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-token\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.919866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.919848    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpsx7\" (UniqueName: \"kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-kube-api-access-hpsx7\") pod \"e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") " pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:04.998074 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:04.998053    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:05.134055 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:05.134030    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"]
Jul 07 11:50:05.135984 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:50:05.135957    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabadecad_01d5_4e69_8856_7c1a2d7ee848.slice/crio-223ceaf799f1829c54711ef47b0580556f4bad38c22ca50eaca6a3e413523fa0 WatchSource:0}: Error finding container 223ceaf799f1829c54711ef47b0580556f4bad38c22ca50eaca6a3e413523fa0: Status 404 returned error can't find the container with id 223ceaf799f1829c54711ef47b0580556f4bad38c22ca50eaca6a3e413523fa0
Jul 07 11:50:05.138074 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:05.138041    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:05.138142 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:05.138108    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:05.138180 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:05.138160    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:05.545879 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:05.545842    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" event={"ID":"abadecad-01d5-4e69-8856-7c1a2d7ee848","Type":"ContainerStarted","Data":"788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7"}
Jul 07 11:50:05.546034 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:05.545896    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" event={"ID":"abadecad-01d5-4e69-8856-7c1a2d7ee848","Type":"ContainerStarted","Data":"223ceaf799f1829c54711ef47b0580556f4bad38c22ca50eaca6a3e413523fa0"}
Jul 07 11:50:05.570097 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:05.570052    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" podStartSLOduration=1.57003633 podStartE2EDuration="1.57003633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:50:05.567671023 +0000 UTC m=+2087.106362440" watchObservedRunningTime="2026-07-07 11:50:05.57003633 +0000 UTC m=+2087.108727741"
Jul 07 11:50:05.999186 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:05.999154    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:07.004021 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.003993    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:07.555761 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.555729    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:07.556566 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.556549    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:07.574154 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.574124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:07.656419 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.656391    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"]
Jul 07 11:50:07.659788 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.659767    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:07.664210 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.662651    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-daa276-serving-cert\""
Jul 07 11:50:07.664210 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.662660    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-fpmsd\""
Jul 07 11:50:07.676245 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.676219    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:07.678865 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.678842    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"]
Jul 07 11:50:07.728133 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.728111    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-maas-api-tls\") pod \"maas-api-e2e-api-a-daa276-597d597f79-vpjlv\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") " pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:07.728266 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.728166    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-openshift-service-ca\") pod \"maas-api-e2e-api-a-daa276-597d597f79-vpjlv\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") " pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:07.728266 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.728221    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9bsd\" (UniqueName: \"kubernetes.io/projected/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-kube-api-access-g9bsd\") pod \"maas-api-e2e-api-a-daa276-597d597f79-vpjlv\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") " pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:07.788119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.788087    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:07.829418 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.829337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-maas-api-tls\") pod \"maas-api-e2e-api-a-daa276-597d597f79-vpjlv\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") " pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:07.829418 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.829394    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-openshift-service-ca\") pod \"maas-api-e2e-api-a-daa276-597d597f79-vpjlv\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") " pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:07.829617 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:07.829490    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-daa276-serving-cert: secret "maas-api-e2e-api-a-daa276-serving-cert" not found
Jul 07 11:50:07.829617 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.829541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9bsd\" (UniqueName: \"kubernetes.io/projected/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-kube-api-access-g9bsd\") pod \"maas-api-e2e-api-a-daa276-597d597f79-vpjlv\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") " pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:07.829617 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:07.829551    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-maas-api-tls podName:3f81339f-5fbe-41b4-b072-e5d54d1c5c95 nodeName:}" failed. No retries permitted until 2026-07-07 11:50:08.32953537 +0000 UTC m=+2089.868226759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-maas-api-tls") pod "maas-api-e2e-api-a-daa276-597d597f79-vpjlv" (UID: "3f81339f-5fbe-41b4-b072-e5d54d1c5c95") : secret "maas-api-e2e-api-a-daa276-serving-cert" not found
Jul 07 11:50:07.830056 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.830038    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-openshift-service-ca\") pod \"maas-api-e2e-api-a-daa276-597d597f79-vpjlv\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") " pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:07.842966 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.842940    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9bsd\" (UniqueName: \"kubernetes.io/projected/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-kube-api-access-g9bsd\") pod \"maas-api-e2e-api-a-daa276-597d597f79-vpjlv\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") " pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:07.951537 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:07.951503    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:08.333885 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:08.333863    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-maas-api-tls\") pod \"maas-api-e2e-api-a-daa276-597d597f79-vpjlv\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") " pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:08.336155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:08.336134    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-maas-api-tls\") pod \"maas-api-e2e-api-a-daa276-597d597f79-vpjlv\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") " pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:08.436382 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:08.436354    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:08.540502 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:08.540466    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:08.576229 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:08.576182    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:08.734372 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:08.734349    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"]
Jul 07 11:50:08.735153 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:50:08.735123    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f81339f_5fbe_41b4_b072_e5d54d1c5c95.slice/crio-7ca94fa6cc99ac0b3a4d584327c5432c33b8ebcb62d3b1debc627b579b2c496e WatchSource:0}: Error finding container 7ca94fa6cc99ac0b3a4d584327c5432c33b8ebcb62d3b1debc627b579b2c496e: Status 404 returned error can't find the container with id 7ca94fa6cc99ac0b3a4d584327c5432c33b8ebcb62d3b1debc627b579b2c496e
Jul 07 11:50:09.034033 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.034005    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:50:09.140556 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.140481    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l72hc\" (UniqueName: \"kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-kube-api-access-l72hc\") pod \"092afb33-c2e8-494f-bb86-10facb71d7cb\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") "
Jul 07 11:50:09.140556 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.140517    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-token\") pod \"092afb33-c2e8-494f-bb86-10facb71d7cb\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") "
Jul 07 11:50:09.140556 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.140554    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-data\") pod \"092afb33-c2e8-494f-bb86-10facb71d7cb\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") "
Jul 07 11:50:09.140821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.140598    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-envoy\") pod \"092afb33-c2e8-494f-bb86-10facb71d7cb\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") "
Jul 07 11:50:09.140821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.140628    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-socket\") pod \"092afb33-c2e8-494f-bb86-10facb71d7cb\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") "
Jul 07 11:50:09.140821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.140653    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-certs\") pod \"092afb33-c2e8-494f-bb86-10facb71d7cb\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") "
Jul 07 11:50:09.140821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.140683    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/092afb33-c2e8-494f-bb86-10facb71d7cb-istiod-ca-cert\") pod \"092afb33-c2e8-494f-bb86-10facb71d7cb\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") "
Jul 07 11:50:09.140821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.140782    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-credential-socket\") pod \"092afb33-c2e8-494f-bb86-10facb71d7cb\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") "
Jul 07 11:50:09.140821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.140814    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-podinfo\") pod \"092afb33-c2e8-494f-bb86-10facb71d7cb\" (UID: \"092afb33-c2e8-494f-bb86-10facb71d7cb\") "
Jul 07 11:50:09.141122 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.141012    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "092afb33-c2e8-494f-bb86-10facb71d7cb" (UID: "092afb33-c2e8-494f-bb86-10facb71d7cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:09.141122 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.141105    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/092afb33-c2e8-494f-bb86-10facb71d7cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "092afb33-c2e8-494f-bb86-10facb71d7cb" (UID: "092afb33-c2e8-494f-bb86-10facb71d7cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:50:09.141249 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.141017    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "092afb33-c2e8-494f-bb86-10facb71d7cb" (UID: "092afb33-c2e8-494f-bb86-10facb71d7cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:09.141249 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.141216    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:09.141249 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.141234    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "092afb33-c2e8-494f-bb86-10facb71d7cb" (UID: "092afb33-c2e8-494f-bb86-10facb71d7cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:09.141480 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.141460    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "092afb33-c2e8-494f-bb86-10facb71d7cb" (UID: "092afb33-c2e8-494f-bb86-10facb71d7cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:09.142776 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.142754    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-kube-api-access-l72hc" (OuterVolumeSpecName: "kube-api-access-l72hc") pod "092afb33-c2e8-494f-bb86-10facb71d7cb" (UID: "092afb33-c2e8-494f-bb86-10facb71d7cb"). InnerVolumeSpecName "kube-api-access-l72hc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:09.143097 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.143070    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "092afb33-c2e8-494f-bb86-10facb71d7cb" (UID: "092afb33-c2e8-494f-bb86-10facb71d7cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:50:09.143405 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.143389    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "092afb33-c2e8-494f-bb86-10facb71d7cb" (UID: "092afb33-c2e8-494f-bb86-10facb71d7cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:09.143474 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.143388    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "092afb33-c2e8-494f-bb86-10facb71d7cb" (UID: "092afb33-c2e8-494f-bb86-10facb71d7cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:09.242271 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.242247    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-credential-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:09.242368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.242273    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-podinfo\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:09.242368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.242288    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l72hc\" (UniqueName: \"kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-kube-api-access-l72hc\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:09.242368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.242301    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-token\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:09.242368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.242313    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-data\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:09.242368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.242325    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-istio-envoy\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:09.242368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.242337    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/092afb33-c2e8-494f-bb86-10facb71d7cb-workload-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:09.242368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.242349    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/092afb33-c2e8-494f-bb86-10facb71d7cb-istiod-ca-cert\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:09.567609 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.567576    2582 generic.go:356] "Generic (PLEG): container finished" podID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerID="9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425" exitCode=0
Jul 07 11:50:09.568050 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.567632    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"
Jul 07 11:50:09.568050 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.567658    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" event={"ID":"092afb33-c2e8-494f-bb86-10facb71d7cb","Type":"ContainerDied","Data":"9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425"}
Jul 07 11:50:09.568050 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.567711    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4" event={"ID":"092afb33-c2e8-494f-bb86-10facb71d7cb","Type":"ContainerDied","Data":"c10f1cc6f2cc9d4724b73fab00ca5a7f5fc5d1570d15ca8c22fd65897eb5d26d"}
Jul 07 11:50:09.568050 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.567736    2582 scope.go:117] "RemoveContainer" containerID="9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425"
Jul 07 11:50:09.569547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.569521    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv" event={"ID":"3f81339f-5fbe-41b4-b072-e5d54d1c5c95","Type":"ContainerStarted","Data":"2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4"}
Jul 07 11:50:09.569658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.569571    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv" event={"ID":"3f81339f-5fbe-41b4-b072-e5d54d1c5c95","Type":"ContainerStarted","Data":"7ca94fa6cc99ac0b3a4d584327c5432c33b8ebcb62d3b1debc627b579b2c496e"}
Jul 07 11:50:09.569736 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.569677    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:09.577922 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.577904    2582 scope.go:117] "RemoveContainer" containerID="d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc"
Jul 07 11:50:09.588305 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.588003    2582 scope.go:117] "RemoveContainer" containerID="9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425"
Jul 07 11:50:09.588958 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:09.588694    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425\": container with ID starting with 9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425 not found: ID does not exist" containerID="9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425"
Jul 07 11:50:09.588958 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.588729    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425"} err="failed to get container status \"9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425\": rpc error: code = NotFound desc = could not find container \"9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425\": container with ID starting with 9a01af8694f97a5a35b4e342119bb95219dc99915375e85448c9ae499d373425 not found: ID does not exist"
Jul 07 11:50:09.588958 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.588755    2582 scope.go:117] "RemoveContainer" containerID="d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc"
Jul 07 11:50:09.589164 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:09.589027    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc\": container with ID starting with d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc not found: ID does not exist" containerID="d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc"
Jul 07 11:50:09.589164 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.589054    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc"} err="failed to get container status \"d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc\": rpc error: code = NotFound desc = could not find container \"d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc\": container with ID starting with d3933a70533c5cd3e4b0dcd79a38a9aebb6e88d6a6820296cb03bb3a92c324bc not found: ID does not exist"
Jul 07 11:50:09.589771 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.589733    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv" podStartSLOduration=2.092274248 podStartE2EDuration="2.589721895s" totalImagesPullingTime="497.447647ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:50:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:50:09.588350801 +0000 UTC m=+2091.127042213" watchObservedRunningTime="2026-07-07 11:50:09.589721895 +0000 UTC m=+2091.128413307"
Jul 07 11:50:09.603993 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.603969    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"]
Jul 07 11:50:09.612650 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:09.612628    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c903aa-openshift-default-579fdb5c69-gz4r4"]
Jul 07 11:50:11.025264 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:11.025223    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" path="/var/lib/kubelet/pods/092afb33-c2e8-494f-bb86-10facb71d7cb/volumes"
Jul 07 11:50:12.088637 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.088607    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:12.528314 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.528280    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:50:12.531886 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.531859    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:12.820836 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.820748    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"]
Jul 07 11:50:12.821337 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.821314    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy"
Jul 07 11:50:12.821467 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.821341    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy"
Jul 07 11:50:12.821467 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.821361    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy"
Jul 07 11:50:12.821467 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.821367    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy"
Jul 07 11:50:12.821467 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.821428    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy"
Jul 07 11:50:12.826096 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.826076    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:12.829917 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.829897    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-626a01-openshift-default-dockercfg-hc95r\""
Jul 07 11:50:12.840732 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.840711    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"]
Jul 07 11:50:12.862944 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.862916    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:12.973404 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.973373    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-certs\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:12.973544 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.973419    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-data\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:12.973544 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.973440    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52916e31-5677-4581-9baf-5b5a120c9176-istiod-ca-cert\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:12.973544 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.973458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-credential-socket\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:12.973544 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.973517    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-istio-token\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:12.973693 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.973578    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-socket\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:12.973693 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.973627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwkv2\" (UniqueName: \"kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-kube-api-access-xwkv2\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:12.973693 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.973656    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-envoy\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:12.973815 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:12.973720    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52916e31-5677-4581-9baf-5b5a120c9176-istio-podinfo\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.007383 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.007355    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:13.075005 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.074931    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52916e31-5677-4581-9baf-5b5a120c9176-istio-podinfo\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075005 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.074981    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-certs\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-data\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075031    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52916e31-5677-4581-9baf-5b5a120c9176-istiod-ca-cert\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-credential-socket\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075062    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-istio-token\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075093    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-socket\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwkv2\" (UniqueName: \"kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-kube-api-access-xwkv2\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075230 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075222    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-envoy\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075560 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075516    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-socket\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075619 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075585    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-credential-socket\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075676 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075631    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-certs\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.075676 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.075650    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-data\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.076137 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.076115    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52916e31-5677-4581-9baf-5b5a120c9176-istiod-ca-cert\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.077578 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.077558    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-envoy\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.077680 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.077622    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52916e31-5677-4581-9baf-5b5a120c9176-istio-podinfo\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.083058 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.083036    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-istio-token\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.083273 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.083254    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwkv2\" (UniqueName: \"kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-kube-api-access-xwkv2\") pod \"e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") " pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.136925 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.136900    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:13.277222 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.277174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"]
Jul 07 11:50:13.278930 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:50:13.278898    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52916e31_5677_4581_9baf_5b5a120c9176.slice/crio-e0a50289375c8d8e125d4118d0cbfd77c93f19ac0e818d47ab423495f355aef5 WatchSource:0}: Error finding container e0a50289375c8d8e125d4118d0cbfd77c93f19ac0e818d47ab423495f355aef5: Status 404 returned error can't find the container with id e0a50289375c8d8e125d4118d0cbfd77c93f19ac0e818d47ab423495f355aef5
Jul 07 11:50:13.281315 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.281279    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:13.281392 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.281347    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:13.281392 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.281378    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:13.595801 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.595701    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" event={"ID":"52916e31-5677-4581-9baf-5b5a120c9176","Type":"ContainerStarted","Data":"08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb"}
Jul 07 11:50:13.595801 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.595771    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" event={"ID":"52916e31-5677-4581-9baf-5b5a120c9176","Type":"ContainerStarted","Data":"e0a50289375c8d8e125d4118d0cbfd77c93f19ac0e818d47ab423495f355aef5"}
Jul 07 11:50:13.618237 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:13.618175    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" podStartSLOduration=1.618159217 podStartE2EDuration="1.618159217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:50:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:50:13.617317139 +0000 UTC m=+2095.156008555" watchObservedRunningTime="2026-07-07 11:50:13.618159217 +0000 UTC m=+2095.156850629"
Jul 07 11:50:14.137568 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:14.137532    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:14.142260 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:14.142236    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:14.600628 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:14.600591    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:14.601618 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:14.601599    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:15.595334 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.595301    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:15.627384 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.627357    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:15.689562 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.689507    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"]
Jul 07 11:50:15.693216 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.690112    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="092afb33-c2e8-494f-bb86-10facb71d7cb" containerName="istio-proxy"
Jul 07 11:50:15.695468 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.695443    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:15.699381 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.699357    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-626a01-serving-cert\""
Jul 07 11:50:15.711977 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.710649    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"]
Jul 07 11:50:15.711977 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.710969    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:15.792888 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.792594    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:15.802227 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.802182    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50169a54-ed0d-4c22-94d0-662b9f03bf92-openshift-service-ca\") pod \"maas-api-e2e-api-b-626a01-555d557875-fvw7g\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") " pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:15.802361 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.802239    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50169a54-ed0d-4c22-94d0-662b9f03bf92-maas-api-tls\") pod \"maas-api-e2e-api-b-626a01-555d557875-fvw7g\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") " pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:15.802361 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.802264    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vqc8\" (UniqueName: \"kubernetes.io/projected/50169a54-ed0d-4c22-94d0-662b9f03bf92-kube-api-access-9vqc8\") pod \"maas-api-e2e-api-b-626a01-555d557875-fvw7g\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") " pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:15.903481 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.903388    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50169a54-ed0d-4c22-94d0-662b9f03bf92-openshift-service-ca\") pod \"maas-api-e2e-api-b-626a01-555d557875-fvw7g\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") " pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:15.903481 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.903435    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50169a54-ed0d-4c22-94d0-662b9f03bf92-maas-api-tls\") pod \"maas-api-e2e-api-b-626a01-555d557875-fvw7g\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") " pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:15.903712 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:15.903539    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-626a01-serving-cert: secret "maas-api-e2e-api-b-626a01-serving-cert" not found
Jul 07 11:50:15.903712 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.903592    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vqc8\" (UniqueName: \"kubernetes.io/projected/50169a54-ed0d-4c22-94d0-662b9f03bf92-kube-api-access-9vqc8\") pod \"maas-api-e2e-api-b-626a01-555d557875-fvw7g\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") " pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:15.903712 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:15.903613    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50169a54-ed0d-4c22-94d0-662b9f03bf92-maas-api-tls podName:50169a54-ed0d-4c22-94d0-662b9f03bf92 nodeName:}" failed. No retries permitted until 2026-07-07 11:50:16.403591399 +0000 UTC m=+2097.942282788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50169a54-ed0d-4c22-94d0-662b9f03bf92-maas-api-tls") pod "maas-api-e2e-api-b-626a01-555d557875-fvw7g" (UID: "50169a54-ed0d-4c22-94d0-662b9f03bf92") : secret "maas-api-e2e-api-b-626a01-serving-cert" not found
Jul 07 11:50:15.904002 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.903981    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50169a54-ed0d-4c22-94d0-662b9f03bf92-openshift-service-ca\") pod \"maas-api-e2e-api-b-626a01-555d557875-fvw7g\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") " pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:15.914739 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.914712    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vqc8\" (UniqueName: \"kubernetes.io/projected/50169a54-ed0d-4c22-94d0-662b9f03bf92-kube-api-access-9vqc8\") pod \"maas-api-e2e-api-b-626a01-555d557875-fvw7g\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") " pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:15.997151 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:15.997117    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:16.234294 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:16.234244    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:16.376823 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:16.376778    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:16.407433 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:16.407402    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50169a54-ed0d-4c22-94d0-662b9f03bf92-maas-api-tls\") pod \"maas-api-e2e-api-b-626a01-555d557875-fvw7g\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") " pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:16.411365 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:16.411348    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50169a54-ed0d-4c22-94d0-662b9f03bf92-maas-api-tls\") pod \"maas-api-e2e-api-b-626a01-555d557875-fvw7g\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") " pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:16.611265 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:16.611147    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:16.772219 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:16.769785    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"]
Jul 07 11:50:16.794453 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:50:16.794359    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50169a54_ed0d_4c22_94d0_662b9f03bf92.slice/crio-78dca41791fabe81f733abb222bcf7f081c4be55a22bd8b634f2ee4197640930 WatchSource:0}: Error finding container 78dca41791fabe81f733abb222bcf7f081c4be55a22bd8b634f2ee4197640930: Status 404 returned error can't find the container with id 78dca41791fabe81f733abb222bcf7f081c4be55a22bd8b634f2ee4197640930
Jul 07 11:50:17.624695 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:17.617935    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g" event={"ID":"50169a54-ed0d-4c22-94d0-662b9f03bf92","Type":"ContainerStarted","Data":"434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106"}
Jul 07 11:50:17.624695 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:17.618601    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:17.624695 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:17.618651    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g" event={"ID":"50169a54-ed0d-4c22-94d0-662b9f03bf92","Type":"ContainerStarted","Data":"78dca41791fabe81f733abb222bcf7f081c4be55a22bd8b634f2ee4197640930"}
Jul 07 11:50:17.641357 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:17.641314    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g" podStartSLOduration=2.310223301 podStartE2EDuration="2.64130079s" totalImagesPullingTime="331.077489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:50:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:50:17.639320353 +0000 UTC m=+2099.178011765" watchObservedRunningTime="2026-07-07 11:50:17.64130079 +0000 UTC m=+2099.179992202"
Jul 07 11:50:19.061209 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:19.061165    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:50:19.065996 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:19.065974    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:50:20.107172 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:20.107138    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:20.634457 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:20.634374    2582 generic.go:356] "Generic (PLEG): container finished" podID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerID="788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7" exitCode=0
Jul 07 11:50:20.634457 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:20.634441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" event={"ID":"abadecad-01d5-4e69-8856-7c1a2d7ee848","Type":"ContainerDied","Data":"788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7"}
Jul 07 11:50:20.634848 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:20.634833    2582 scope.go:117] "RemoveContainer" containerID="788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7"
Jul 07 11:50:20.635947 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:20.635920    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:20.636033 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:20.636018    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:20.636076 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:20.636054    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:21.640935 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:21.640897    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" event={"ID":"abadecad-01d5-4e69-8856-7c1a2d7ee848","Type":"ContainerStarted","Data":"bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8"}
Jul 07 11:50:21.999090 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:21.999047    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:22.000309 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:22.000281    2582 patch_prober.go:28] interesting pod/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 07 11:50:22.000429 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:22.000349    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 07 11:50:22.556563 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:22.556517    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:22.999184 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:22.999135    2582 patch_prober.go:28] interesting pod/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 07 11:50:22.999760 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:22.999246    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 07 11:50:23.629087 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:23.629059    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:23.998455 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:23.998410    2582 patch_prober.go:28] interesting pod/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 07 11:50:23.998670 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:23.998476    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 07 11:50:24.998958 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:24.998917    2582 patch_prober.go:28] interesting pod/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 07 11:50:24.999362 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:24.998978    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 07 11:50:25.998617 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:25.998566    2582 patch_prober.go:28] interesting pod/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 07 11:50:25.998833 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:25.998633    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 07 11:50:27.003019 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:27.002985    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:27.004228 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:27.004179    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:28.180369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.180287    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:28.231902 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.231870    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:50:28.562601 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.562561    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:28.659174 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.659137    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:28.677716 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.677685    2582 generic.go:356] "Generic (PLEG): container finished" podID="52916e31-5677-4581-9baf-5b5a120c9176" containerID="08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb" exitCode=0
Jul 07 11:50:28.677848 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.677758    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" event={"ID":"52916e31-5677-4581-9baf-5b5a120c9176","Type":"ContainerDied","Data":"08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb"}
Jul 07 11:50:28.678152 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.678129    2582 scope.go:117] "RemoveContainer" containerID="08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb"
Jul 07 11:50:28.679143 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.679116    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:28.679270 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.679216    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:28.679270 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.679252    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:28.758652 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.758608    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:28.868607 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.863487    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"]
Jul 07 11:50:28.868607 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.863764    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv" podUID="3f81339f-5fbe-41b4-b072-e5d54d1c5c95" containerName="maas-api" containerID="cri-o://2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4" gracePeriod=30
Jul 07 11:50:28.897529 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:28.897504    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:29.124491 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.124434    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:29.227567 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.227534    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-openshift-service-ca\") pod \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") "
Jul 07 11:50:29.227998 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.227582    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-maas-api-tls\") pod \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") "
Jul 07 11:50:29.227998 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.227668    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9bsd\" (UniqueName: \"kubernetes.io/projected/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-kube-api-access-g9bsd\") pod \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\" (UID: \"3f81339f-5fbe-41b4-b072-e5d54d1c5c95\") "
Jul 07 11:50:29.227998 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.227891    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3f81339f-5fbe-41b4-b072-e5d54d1c5c95" (UID: "3f81339f-5fbe-41b4-b072-e5d54d1c5c95"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:50:29.229723 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.229693    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-kube-api-access-g9bsd" (OuterVolumeSpecName: "kube-api-access-g9bsd") pod "3f81339f-5fbe-41b4-b072-e5d54d1c5c95" (UID: "3f81339f-5fbe-41b4-b072-e5d54d1c5c95"). InnerVolumeSpecName "kube-api-access-g9bsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:29.229836 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.229741    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3f81339f-5fbe-41b4-b072-e5d54d1c5c95" (UID: "3f81339f-5fbe-41b4-b072-e5d54d1c5c95"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:50:29.328540 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.328510    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:29.328540 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.328537    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:29.328540 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.328546    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9bsd\" (UniqueName: \"kubernetes.io/projected/3f81339f-5fbe-41b4-b072-e5d54d1c5c95-kube-api-access-g9bsd\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:29.600623 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.600592    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:29.683924 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.683885    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" event={"ID":"52916e31-5677-4581-9baf-5b5a120c9176","Type":"ContainerStarted","Data":"7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29"}
Jul 07 11:50:29.685173 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.685148    2582 generic.go:356] "Generic (PLEG): container finished" podID="3f81339f-5fbe-41b4-b072-e5d54d1c5c95" containerID="2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4" exitCode=0
Jul 07 11:50:29.685324 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.685189    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv" event={"ID":"3f81339f-5fbe-41b4-b072-e5d54d1c5c95","Type":"ContainerDied","Data":"2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4"}
Jul 07 11:50:29.685324 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.685242    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv" event={"ID":"3f81339f-5fbe-41b4-b072-e5d54d1c5c95","Type":"ContainerDied","Data":"7ca94fa6cc99ac0b3a4d584327c5432c33b8ebcb62d3b1debc627b579b2c496e"}
Jul 07 11:50:29.685324 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.685265    2582 scope.go:117] "RemoveContainer" containerID="2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4"
Jul 07 11:50:29.685324 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.685211    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"
Jul 07 11:50:29.694948 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.694929    2582 scope.go:117] "RemoveContainer" containerID="2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4"
Jul 07 11:50:29.695294 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:29.695185    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4\": container with ID starting with 2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4 not found: ID does not exist" containerID="2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4"
Jul 07 11:50:29.695368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.695305    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4"} err="failed to get container status \"2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4\": rpc error: code = NotFound desc = could not find container \"2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4\": container with ID starting with 2ab7e9e25de62df46a97f4fd1d8c90af776535a55017c664e10252d575360ac4 not found: ID does not exist"
Jul 07 11:50:29.719956 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.719933    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"]
Jul 07 11:50:29.727158 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:29.727135    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-daa276-597d597f79-vpjlv"]
Jul 07 11:50:30.137190 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:30.137150    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:30.138224 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:30.138184    2582 patch_prober.go:28] interesting pod/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 11:50:30.138352 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:30.138256    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 11:50:31.025675 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:31.025641    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f81339f-5fbe-41b4-b072-e5d54d1c5c95" path="/var/lib/kubelet/pods/3f81339f-5fbe-41b4-b072-e5d54d1c5c95/volumes"
Jul 07 11:50:31.137621 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:31.137583    2582 patch_prober.go:28] interesting pod/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 11:50:31.137790 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:31.137639    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 11:50:32.137872 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:32.137835    2582 patch_prober.go:28] interesting pod/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 11:50:32.138285 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:32.137894    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 11:50:33.137743 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:33.137708    2582 patch_prober.go:28] interesting pod/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 11:50:33.137925 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:33.137763    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 11:50:34.142358 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:34.142321    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:34.142719 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:34.142641    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:34.143467 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:34.143452    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:34.424915 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:34.424829    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:34.525300 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:34.525238    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:34.679945 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:34.679869    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"]
Jul 07 11:50:34.680119 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:34.680097    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy" containerID="cri-o://bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8" gracePeriod=30
Jul 07 11:50:34.690352 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:34.690329    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:34.919658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:34.919628    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:35.070091 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:35.070051    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:35.479352 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:35.479316    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:35.581083 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:35.581050    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:35.678017 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:35.677982    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:35.771877 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:35.771785    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"]
Jul 07 11:50:35.772078 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:35.772024    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g" podUID="50169a54-ed0d-4c22-94d0-662b9f03bf92" containerName="maas-api" containerID="cri-o://434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106" gracePeriod=30
Jul 07 11:50:35.798438 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:35.798410    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:36.030936 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.030872    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:36.114547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.114518    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50169a54-ed0d-4c22-94d0-662b9f03bf92-openshift-service-ca\") pod \"50169a54-ed0d-4c22-94d0-662b9f03bf92\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") "
Jul 07 11:50:36.114547 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.114552    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50169a54-ed0d-4c22-94d0-662b9f03bf92-maas-api-tls\") pod \"50169a54-ed0d-4c22-94d0-662b9f03bf92\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") "
Jul 07 11:50:36.114769 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.114579    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vqc8\" (UniqueName: \"kubernetes.io/projected/50169a54-ed0d-4c22-94d0-662b9f03bf92-kube-api-access-9vqc8\") pod \"50169a54-ed0d-4c22-94d0-662b9f03bf92\" (UID: \"50169a54-ed0d-4c22-94d0-662b9f03bf92\") "
Jul 07 11:50:36.115029 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.114999    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50169a54-ed0d-4c22-94d0-662b9f03bf92-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50169a54-ed0d-4c22-94d0-662b9f03bf92" (UID: "50169a54-ed0d-4c22-94d0-662b9f03bf92"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:50:36.116702 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.116677    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50169a54-ed0d-4c22-94d0-662b9f03bf92-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50169a54-ed0d-4c22-94d0-662b9f03bf92" (UID: "50169a54-ed0d-4c22-94d0-662b9f03bf92"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:50:36.116802 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.116743    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50169a54-ed0d-4c22-94d0-662b9f03bf92-kube-api-access-9vqc8" (OuterVolumeSpecName: "kube-api-access-9vqc8") pod "50169a54-ed0d-4c22-94d0-662b9f03bf92" (UID: "50169a54-ed0d-4c22-94d0-662b9f03bf92"). InnerVolumeSpecName "kube-api-access-9vqc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:36.216013 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.215981    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50169a54-ed0d-4c22-94d0-662b9f03bf92-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:36.216013 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.216010    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50169a54-ed0d-4c22-94d0-662b9f03bf92-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:36.216013 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.216020    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vqc8\" (UniqueName: \"kubernetes.io/projected/50169a54-ed0d-4c22-94d0-662b9f03bf92-kube-api-access-9vqc8\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:36.720757 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.720728    2582 generic.go:356] "Generic (PLEG): container finished" podID="50169a54-ed0d-4c22-94d0-662b9f03bf92" containerID="434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106" exitCode=0
Jul 07 11:50:36.721147 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.720776    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"
Jul 07 11:50:36.721147 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.720778    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g" event={"ID":"50169a54-ed0d-4c22-94d0-662b9f03bf92","Type":"ContainerDied","Data":"434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106"}
Jul 07 11:50:36.721147 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.720901    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g" event={"ID":"50169a54-ed0d-4c22-94d0-662b9f03bf92","Type":"ContainerDied","Data":"78dca41791fabe81f733abb222bcf7f081c4be55a22bd8b634f2ee4197640930"}
Jul 07 11:50:36.721147 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.720933    2582 scope.go:117] "RemoveContainer" containerID="434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106"
Jul 07 11:50:36.729915 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.729895    2582 scope.go:117] "RemoveContainer" containerID="434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106"
Jul 07 11:50:36.730155 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:36.730137    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106\": container with ID starting with 434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106 not found: ID does not exist" containerID="434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106"
Jul 07 11:50:36.730224 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.730164    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106"} err="failed to get container status \"434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106\": rpc error: code = NotFound desc = could not find container \"434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106\": container with ID starting with 434ca5e35b6f227dc8a8db91abac22ef4f424040c3b87a541cd056f2fd8bd106 not found: ID does not exist"
Jul 07 11:50:36.742316 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.742294    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"]
Jul 07 11:50:36.746902 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:36.746880    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-626a01-555d557875-fvw7g"]
Jul 07 11:50:37.025082 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:37.025003    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50169a54-ed0d-4c22-94d0-662b9f03bf92" path="/var/lib/kubelet/pods/50169a54-ed0d-4c22-94d0-662b9f03bf92/volumes"
Jul 07 11:50:39.125233 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:39.125189    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:50:39.932606 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:39.932583    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:40.048926 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.048898    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-token\") pod \"abadecad-01d5-4e69-8856-7c1a2d7ee848\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") "
Jul 07 11:50:40.049115 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.048942    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-certs\") pod \"abadecad-01d5-4e69-8856-7c1a2d7ee848\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") "
Jul 07 11:50:40.049115 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.048960    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-credential-socket\") pod \"abadecad-01d5-4e69-8856-7c1a2d7ee848\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") "
Jul 07 11:50:40.049115 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.048978    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-podinfo\") pod \"abadecad-01d5-4e69-8856-7c1a2d7ee848\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") "
Jul 07 11:50:40.049115 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049002    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-envoy\") pod \"abadecad-01d5-4e69-8856-7c1a2d7ee848\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") "
Jul 07 11:50:40.049115 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049036    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abadecad-01d5-4e69-8856-7c1a2d7ee848-istiod-ca-cert\") pod \"abadecad-01d5-4e69-8856-7c1a2d7ee848\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") "
Jul 07 11:50:40.049115 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049060    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpsx7\" (UniqueName: \"kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-kube-api-access-hpsx7\") pod \"abadecad-01d5-4e69-8856-7c1a2d7ee848\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") "
Jul 07 11:50:40.049527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049126    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-data\") pod \"abadecad-01d5-4e69-8856-7c1a2d7ee848\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") "
Jul 07 11:50:40.049527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049151    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-socket\") pod \"abadecad-01d5-4e69-8856-7c1a2d7ee848\" (UID: \"abadecad-01d5-4e69-8856-7c1a2d7ee848\") "
Jul 07 11:50:40.049527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049215    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "abadecad-01d5-4e69-8856-7c1a2d7ee848" (UID: "abadecad-01d5-4e69-8856-7c1a2d7ee848"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:40.049527 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049328    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "abadecad-01d5-4e69-8856-7c1a2d7ee848" (UID: "abadecad-01d5-4e69-8856-7c1a2d7ee848"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:40.049736 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049528    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abadecad-01d5-4e69-8856-7c1a2d7ee848-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "abadecad-01d5-4e69-8856-7c1a2d7ee848" (UID: "abadecad-01d5-4e69-8856-7c1a2d7ee848"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:50:40.049736 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049606    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:40.049736 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049627    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-credential-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:40.050010 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.049983    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "abadecad-01d5-4e69-8856-7c1a2d7ee848" (UID: "abadecad-01d5-4e69-8856-7c1a2d7ee848"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:40.050584 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.050555    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-data" (OuterVolumeSpecName: "istio-data") pod "abadecad-01d5-4e69-8856-7c1a2d7ee848" (UID: "abadecad-01d5-4e69-8856-7c1a2d7ee848"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:40.051443 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.051416    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "abadecad-01d5-4e69-8856-7c1a2d7ee848" (UID: "abadecad-01d5-4e69-8856-7c1a2d7ee848"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:50:40.051667 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.051647    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-token" (OuterVolumeSpecName: "istio-token") pod "abadecad-01d5-4e69-8856-7c1a2d7ee848" (UID: "abadecad-01d5-4e69-8856-7c1a2d7ee848"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:40.051667 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.051647    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "abadecad-01d5-4e69-8856-7c1a2d7ee848" (UID: "abadecad-01d5-4e69-8856-7c1a2d7ee848"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:40.051761 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.051722    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-kube-api-access-hpsx7" (OuterVolumeSpecName: "kube-api-access-hpsx7") pod "abadecad-01d5-4e69-8856-7c1a2d7ee848" (UID: "abadecad-01d5-4e69-8856-7c1a2d7ee848"). InnerVolumeSpecName "kube-api-access-hpsx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:40.150691 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.150660    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-envoy\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:40.150691 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.150686    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abadecad-01d5-4e69-8856-7c1a2d7ee848-istiod-ca-cert\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:40.151024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.150695    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpsx7\" (UniqueName: \"kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-kube-api-access-hpsx7\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:40.151024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.150704    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-data\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:40.151024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.150713    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abadecad-01d5-4e69-8856-7c1a2d7ee848-workload-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:40.151024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.150721    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-token\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:40.151024 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.150729    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abadecad-01d5-4e69-8856-7c1a2d7ee848-istio-podinfo\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:40.741419 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.741383    2582 generic.go:356] "Generic (PLEG): container finished" podID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerID="bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8" exitCode=0
Jul 07 11:50:40.741587 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.741434    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"
Jul 07 11:50:40.741587 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.741462    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" event={"ID":"abadecad-01d5-4e69-8856-7c1a2d7ee848","Type":"ContainerDied","Data":"bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8"}
Jul 07 11:50:40.741587 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.741518    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw" event={"ID":"abadecad-01d5-4e69-8856-7c1a2d7ee848","Type":"ContainerDied","Data":"223ceaf799f1829c54711ef47b0580556f4bad38c22ca50eaca6a3e413523fa0"}
Jul 07 11:50:40.741587 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.741540    2582 scope.go:117] "RemoveContainer" containerID="bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8"
Jul 07 11:50:40.750795 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.750779    2582 scope.go:117] "RemoveContainer" containerID="788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7"
Jul 07 11:50:40.758736 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.758720    2582 scope.go:117] "RemoveContainer" containerID="bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8"
Jul 07 11:50:40.758960 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:40.758944    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8\": container with ID starting with bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8 not found: ID does not exist" containerID="bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8"
Jul 07 11:50:40.759006 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.758967    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8"} err="failed to get container status \"bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8\": rpc error: code = NotFound desc = could not find container \"bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8\": container with ID starting with bd579294463fb661eb0276086b939425c77d421bb0222e10f1b82d974497e8d8 not found: ID does not exist"
Jul 07 11:50:40.759006 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.758984    2582 scope.go:117] "RemoveContainer" containerID="788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7"
Jul 07 11:50:40.759236 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:40.759217    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7\": container with ID starting with 788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7 not found: ID does not exist" containerID="788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7"
Jul 07 11:50:40.759276 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.759243    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7"} err="failed to get container status \"788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7\": rpc error: code = NotFound desc = could not find container \"788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7\": container with ID starting with 788acb007513a97324648009392f38b129fe0de3646f4bc3385e510aa0bc8dc7 not found: ID does not exist"
Jul 07 11:50:40.763868 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.763845    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"]
Jul 07 11:50:40.768212 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:40.768180    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-daa276-openshift-default-65bcddcc-hcgmw"]
Jul 07 11:50:41.025928 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:41.025844    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" path="/var/lib/kubelet/pods/abadecad-01d5-4e69-8856-7c1a2d7ee848/volumes"
Jul 07 11:50:41.679272 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:41.679235    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:41.775214 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:41.775174    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:41.917076 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:41.917044    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"]
Jul 07 11:50:41.917310 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:41.917287    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy" containerID="cri-o://7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29" gracePeriod=30
Jul 07 11:50:41.940718 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:41.940640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:42.170623 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.170591    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:42.507481 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.507449    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:42.784876 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.784782    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"]
Jul 07 11:50:42.785328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.785127    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy"
Jul 07 11:50:42.785328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.785139    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy"
Jul 07 11:50:42.785328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.785157    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f81339f-5fbe-41b4-b072-e5d54d1c5c95" containerName="maas-api"
Jul 07 11:50:42.785328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.785163    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f81339f-5fbe-41b4-b072-e5d54d1c5c95" containerName="maas-api"
Jul 07 11:50:42.785328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.785169    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50169a54-ed0d-4c22-94d0-662b9f03bf92" containerName="maas-api"
Jul 07 11:50:42.785328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.785174    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="50169a54-ed0d-4c22-94d0-662b9f03bf92" containerName="maas-api"
Jul 07 11:50:42.785328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.785237    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy"
Jul 07 11:50:42.785328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.785247    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="50169a54-ed0d-4c22-94d0-662b9f03bf92" containerName="maas-api"
Jul 07 11:50:42.785328 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.785252    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f81339f-5fbe-41b4-b072-e5d54d1c5c95" containerName="maas-api"
Jul 07 11:50:42.789670 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.789650    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.792781 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.792762    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-ab56d6-openshift-default-dockercfg-gtcm9\""
Jul 07 11:50:42.800580 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.800561    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"]
Jul 07 11:50:42.813148 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.813120    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:42.874206 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.874177    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-token\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.874308 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.874226    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-data\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.874308 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.874252    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-certs\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.874308 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.874282    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-socket\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.874442 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.874307    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-podinfo\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.874442 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.874363    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-envoy\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.874442 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.874387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-credential-socket\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.874442 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.874405    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6c61125-4862-4011-99f0-cf8307f6e4a6-istiod-ca-cert\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.874442 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.874421    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjv6f\" (UniqueName: \"kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-kube-api-access-jjv6f\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.975622 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.975592    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-token\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.975723 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.975642    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-data\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.975797 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.975768    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-certs\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.975856 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.975836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-socket\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.975912 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.975878    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-podinfo\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.975972 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.975920    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-envoy\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.975972 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.975954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-credential-socket\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.976073 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.975983    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6c61125-4862-4011-99f0-cf8307f6e4a6-istiod-ca-cert\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.976073 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.976008    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjv6f\" (UniqueName: \"kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-kube-api-access-jjv6f\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.976172 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.976096    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-certs\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.976172 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.976012    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-data\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.976172 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.976166    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-socket\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.976396 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.976255    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-credential-socket\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.976737 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.976716    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6c61125-4862-4011-99f0-cf8307f6e4a6-istiod-ca-cert\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.978255 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.978237    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-envoy\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.978785 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.978765    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-podinfo\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.985128 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.985105    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-token\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:42.988304 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:42.988281    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjv6f\" (UniqueName: \"kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-kube-api-access-jjv6f\") pod \"e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") " pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:43.101113 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:43.101045    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:43.189608 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:43.189499    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:43.241085 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:43.241060    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"]
Jul 07 11:50:43.243552 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:50:43.243529    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6c61125_4862_4011_99f0_cf8307f6e4a6.slice/crio-0285fe844ad9a658d3061dce50515676e19bb1b5464aaf2c0e994459f2284f1b WatchSource:0}: Error finding container 0285fe844ad9a658d3061dce50515676e19bb1b5464aaf2c0e994459f2284f1b: Status 404 returned error can't find the container with id 0285fe844ad9a658d3061dce50515676e19bb1b5464aaf2c0e994459f2284f1b
Jul 07 11:50:43.245604 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:43.245569    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:43.245686 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:43.245636    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:43.245686 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:43.245668    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:43.757790 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:43.757747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" event={"ID":"e6c61125-4862-4011-99f0-cf8307f6e4a6","Type":"ContainerStarted","Data":"2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33"}
Jul 07 11:50:43.757990 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:43.757809    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" event={"ID":"e6c61125-4862-4011-99f0-cf8307f6e4a6","Type":"ContainerStarted","Data":"0285fe844ad9a658d3061dce50515676e19bb1b5464aaf2c0e994459f2284f1b"}
Jul 07 11:50:43.781252 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:43.781177    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" podStartSLOduration=1.781162794 podStartE2EDuration="1.781162794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:50:43.780259296 +0000 UTC m=+2125.318950708" watchObservedRunningTime="2026-07-07 11:50:43.781162794 +0000 UTC m=+2125.319854206"
Jul 07 11:50:44.101595 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:44.101504    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:44.102889 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:44.102866    2582 patch_prober.go:28] interesting pod/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 07 11:50:44.102953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:44.102915    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 07 11:50:45.105499 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.105471    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:45.769153 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.769119    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:45.770258 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.770239    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:45.874298 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.874268    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:45.950823 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.950789    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"]
Jul 07 11:50:45.951381 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.951362    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy"
Jul 07 11:50:45.951436 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.951386    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy"
Jul 07 11:50:45.951501 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.951489    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="abadecad-01d5-4e69-8856-7c1a2d7ee848" containerName="istio-proxy"
Jul 07 11:50:45.953490 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.953456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:45.954922 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.954902    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:45.957674 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.957656    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-fpmsd\""
Jul 07 11:50:45.957857 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.957845    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-ab56d6-serving-cert\""
Jul 07 11:50:45.964035 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:45.964014    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"]
Jul 07 11:50:46.003299 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.003271    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9fn5\" (UniqueName: \"kubernetes.io/projected/b4a12624-1229-426a-8b93-08fa98d4be01-kube-api-access-g9fn5\") pod \"maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") " pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:46.003432 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.003308    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a12624-1229-426a-8b93-08fa98d4be01-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") " pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:46.003432 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.003353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a12624-1229-426a-8b93-08fa98d4be01-maas-api-tls\") pod \"maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") " pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:46.015974 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.015943    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:46.104853 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.104755    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9fn5\" (UniqueName: \"kubernetes.io/projected/b4a12624-1229-426a-8b93-08fa98d4be01-kube-api-access-g9fn5\") pod \"maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") " pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:46.104853 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.104791    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a12624-1229-426a-8b93-08fa98d4be01-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") " pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:46.105095 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.104880    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a12624-1229-426a-8b93-08fa98d4be01-maas-api-tls\") pod \"maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") " pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:46.105095 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:46.105013    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-ab56d6-serving-cert: secret "maas-api-e2e-shared-a-ab56d6-serving-cert" not found
Jul 07 11:50:46.105095 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:46.105081    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4a12624-1229-426a-8b93-08fa98d4be01-maas-api-tls podName:b4a12624-1229-426a-8b93-08fa98d4be01 nodeName:}" failed. No retries permitted until 2026-07-07 11:50:46.605058868 +0000 UTC m=+2128.143750260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b4a12624-1229-426a-8b93-08fa98d4be01-maas-api-tls") pod "maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899" (UID: "b4a12624-1229-426a-8b93-08fa98d4be01") : secret "maas-api-e2e-shared-a-ab56d6-serving-cert" not found
Jul 07 11:50:46.105475 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.105455    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a12624-1229-426a-8b93-08fa98d4be01-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") " pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:46.115906 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.115878    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9fn5\" (UniqueName: \"kubernetes.io/projected/b4a12624-1229-426a-8b93-08fa98d4be01-kube-api-access-g9fn5\") pod \"maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") " pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:46.185235 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.185175    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:46.572645 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.572603    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:46.608776 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.608742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a12624-1229-426a-8b93-08fa98d4be01-maas-api-tls\") pod \"maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") " pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:46.611128 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.611101    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a12624-1229-426a-8b93-08fa98d4be01-maas-api-tls\") pod \"maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") " pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:46.672387 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.672359    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:46.867840 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:46.867751    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:47.006602 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.006576    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"]
Jul 07 11:50:47.008255 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:50:47.008224    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a12624_1229_426a_8b93_08fa98d4be01.slice/crio-4e5a7ad2c343174581f411acacd5ac938cfe5aff3742c035cdac568e4143d96b WatchSource:0}: Error finding container 4e5a7ad2c343174581f411acacd5ac938cfe5aff3742c035cdac568e4143d96b: Status 404 returned error can't find the container with id 4e5a7ad2c343174581f411acacd5ac938cfe5aff3742c035cdac568e4143d96b
Jul 07 11:50:47.170207 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.170167    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:47.213656 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.213627    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-credential-socket\") pod \"52916e31-5677-4581-9baf-5b5a120c9176\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") "
Jul 07 11:50:47.213656 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.213658    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwkv2\" (UniqueName: \"kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-kube-api-access-xwkv2\") pod \"52916e31-5677-4581-9baf-5b5a120c9176\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") "
Jul 07 11:50:47.213840 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.213698    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-data\") pod \"52916e31-5677-4581-9baf-5b5a120c9176\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") "
Jul 07 11:50:47.213840 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.213739    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52916e31-5677-4581-9baf-5b5a120c9176-istiod-ca-cert\") pod \"52916e31-5677-4581-9baf-5b5a120c9176\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") "
Jul 07 11:50:47.213840 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.213774    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-certs\") pod \"52916e31-5677-4581-9baf-5b5a120c9176\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") "
Jul 07 11:50:47.213954 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.213905    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "52916e31-5677-4581-9baf-5b5a120c9176" (UID: "52916e31-5677-4581-9baf-5b5a120c9176"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:47.214016 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.213963    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52916e31-5677-4581-9baf-5b5a120c9176-istio-podinfo\") pod \"52916e31-5677-4581-9baf-5b5a120c9176\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") "
Jul 07 11:50:47.214155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214123    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-socket\") pod \"52916e31-5677-4581-9baf-5b5a120c9176\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") "
Jul 07 11:50:47.214155 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214152    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-istio-token\") pod \"52916e31-5677-4581-9baf-5b5a120c9176\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") "
Jul 07 11:50:47.214355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214177    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-envoy\") pod \"52916e31-5677-4581-9baf-5b5a120c9176\" (UID: \"52916e31-5677-4581-9baf-5b5a120c9176\") "
Jul 07 11:50:47.214355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214218    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52916e31-5677-4581-9baf-5b5a120c9176-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "52916e31-5677-4581-9baf-5b5a120c9176" (UID: "52916e31-5677-4581-9baf-5b5a120c9176"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:50:47.214355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214215    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "52916e31-5677-4581-9baf-5b5a120c9176" (UID: "52916e31-5677-4581-9baf-5b5a120c9176"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:47.214522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214421    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "52916e31-5677-4581-9baf-5b5a120c9176" (UID: "52916e31-5677-4581-9baf-5b5a120c9176"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:47.214522 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214507    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:47.214627 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214525    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-credential-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:47.214627 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214539    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52916e31-5677-4581-9baf-5b5a120c9176-istiod-ca-cert\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:47.214627 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214554    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-workload-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:47.214788 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.214668    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-data" (OuterVolumeSpecName: "istio-data") pod "52916e31-5677-4581-9baf-5b5a120c9176" (UID: "52916e31-5677-4581-9baf-5b5a120c9176"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:47.216131 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.216091    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-kube-api-access-xwkv2" (OuterVolumeSpecName: "kube-api-access-xwkv2") pod "52916e31-5677-4581-9baf-5b5a120c9176" (UID: "52916e31-5677-4581-9baf-5b5a120c9176"). InnerVolumeSpecName "kube-api-access-xwkv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:47.216410 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.216387    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "52916e31-5677-4581-9baf-5b5a120c9176" (UID: "52916e31-5677-4581-9baf-5b5a120c9176"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:47.216658 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.216638    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-istio-token" (OuterVolumeSpecName: "istio-token") pod "52916e31-5677-4581-9baf-5b5a120c9176" (UID: "52916e31-5677-4581-9baf-5b5a120c9176"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:47.216734 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.216655    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52916e31-5677-4581-9baf-5b5a120c9176-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "52916e31-5677-4581-9baf-5b5a120c9176" (UID: "52916e31-5677-4581-9baf-5b5a120c9176"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:50:47.315991 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.315951    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-data\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:47.315991 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.315983    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52916e31-5677-4581-9baf-5b5a120c9176-istio-podinfo\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:47.315991 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.315992    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-istio-token\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:47.315991 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.316001    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52916e31-5677-4581-9baf-5b5a120c9176-istio-envoy\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:47.316369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.316010    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwkv2\" (UniqueName: \"kubernetes.io/projected/52916e31-5677-4581-9baf-5b5a120c9176-kube-api-access-xwkv2\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:50:47.780892 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.780856    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899" event={"ID":"b4a12624-1229-426a-8b93-08fa98d4be01","Type":"ContainerStarted","Data":"cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24"}
Jul 07 11:50:47.781109 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.780924    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:47.781109 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.780952    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899" event={"ID":"b4a12624-1229-426a-8b93-08fa98d4be01","Type":"ContainerStarted","Data":"4e5a7ad2c343174581f411acacd5ac938cfe5aff3742c035cdac568e4143d96b"}
Jul 07 11:50:47.782652 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.782620    2582 generic.go:356] "Generic (PLEG): container finished" podID="52916e31-5677-4581-9baf-5b5a120c9176" containerID="7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29" exitCode=0
Jul 07 11:50:47.782785 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.782662    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"
Jul 07 11:50:47.782785 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.782696    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" event={"ID":"52916e31-5677-4581-9baf-5b5a120c9176","Type":"ContainerDied","Data":"7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29"}
Jul 07 11:50:47.782785 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.782744    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c" event={"ID":"52916e31-5677-4581-9baf-5b5a120c9176","Type":"ContainerDied","Data":"e0a50289375c8d8e125d4118d0cbfd77c93f19ac0e818d47ab423495f355aef5"}
Jul 07 11:50:47.782785 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.782765    2582 scope.go:117] "RemoveContainer" containerID="7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29"
Jul 07 11:50:47.792176 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.792159    2582 scope.go:117] "RemoveContainer" containerID="08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb"
Jul 07 11:50:47.801699 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.801678    2582 scope.go:117] "RemoveContainer" containerID="7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29"
Jul 07 11:50:47.801945 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:47.801926    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29\": container with ID starting with 7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29 not found: ID does not exist" containerID="7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29"
Jul 07 11:50:47.802008 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.801951    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29"} err="failed to get container status \"7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29\": rpc error: code = NotFound desc = could not find container \"7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29\": container with ID starting with 7e5f17d8cd68ecdfd0a79539e3eba5c551bdc277f553993da6025666a4c78c29 not found: ID does not exist"
Jul 07 11:50:47.802008 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.801968    2582 scope.go:117] "RemoveContainer" containerID="08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb"
Jul 07 11:50:47.802229 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:47.802206    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb\": container with ID starting with 08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb not found: ID does not exist" containerID="08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb"
Jul 07 11:50:47.802289 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.802234    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb"} err="failed to get container status \"08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb\": rpc error: code = NotFound desc = could not find container \"08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb\": container with ID starting with 08c8f0a5603c99991c9ae7e83c21c218bbdc1b875a4e7be63f53e1a3cca191eb not found: ID does not exist"
Jul 07 11:50:47.869025 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.868978    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899" podStartSLOduration=2.433098876 podStartE2EDuration="2.86896426s" totalImagesPullingTime="435.865384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:50:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:50:47.836794204 +0000 UTC m=+2129.375485615" watchObservedRunningTime="2026-07-07 11:50:47.86896426 +0000 UTC m=+2129.407655673"
Jul 07 11:50:47.870762 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.870743    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"]
Jul 07 11:50:47.883678 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:47.883652    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-626a01-openshift-default-774dd87bbf-g5l8c"]
Jul 07 11:50:49.024939 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:49.024904    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52916e31-5677-4581-9baf-5b5a120c9176" path="/var/lib/kubelet/pods/52916e31-5677-4581-9baf-5b5a120c9176/volumes"
Jul 07 11:50:50.117554 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.117525    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:50:50.270902 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.270871    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:50.599857 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.599820    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:50.890278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.885581    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"]
Jul 07 11:50:50.890278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.886105    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy"
Jul 07 11:50:50.890278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.886121    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy"
Jul 07 11:50:50.890278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.886132    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy"
Jul 07 11:50:50.890278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.886141    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy"
Jul 07 11:50:50.890278 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.886274    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy"
Jul 07 11:50:50.892502 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.891095    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:50.896731 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.896563    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-97eb93-openshift-default-dockercfg-wthdj\""
Jul 07 11:50:50.909745 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.909713    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"]
Jul 07 11:50:50.917543 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.917518    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:50.947592 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.947568    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-data\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:50.947700 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.947594    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istiod-ca-cert\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:50.947700 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.947645    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-certs\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:50.947700 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.947671    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-envoy\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:50.947819 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.947703    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-socket\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:50.947819 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.947753    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gnjk\" (UniqueName: \"kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-kube-api-access-8gnjk\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:50.947819 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.947781    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-credential-socket\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:50.947819 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.947817    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-podinfo\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:50.947971 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:50.947836    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-token\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.017483 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.017445    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:51.048453 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048424    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-socket\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.048610 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048473    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gnjk\" (UniqueName: \"kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-kube-api-access-8gnjk\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.048610 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-credential-socket\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.048610 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048545    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-podinfo\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.048610 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-token\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.048770 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048744    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-data\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.048821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048774    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istiod-ca-cert\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.048884 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048865    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-credential-socket\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.048940 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048887    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-certs\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.048940 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048917    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-socket\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.049046 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.048936    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-envoy\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.049136 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.049113    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-data\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.049221 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.049151    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-certs\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.049473 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.049447    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istiod-ca-cert\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.050983 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.050964    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-envoy\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.051065 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.051048    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-podinfo\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.056723 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.056701    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-token\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.056919 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.056903    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gnjk\" (UniqueName: \"kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-kube-api-access-8gnjk\") pod \"e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") " pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.212009 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.211926    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:51.385232 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.382827    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"]
Jul 07 11:50:51.395478 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.395447    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:51.395583 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.395513    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:51.395583 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.395540    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:51.815170 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.815133    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" event={"ID":"d4f3cabc-017f-4773-be7e-02844a5c9dbb","Type":"ContainerStarted","Data":"045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8"}
Jul 07 11:50:51.815393 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.815186    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" event={"ID":"d4f3cabc-017f-4773-be7e-02844a5c9dbb","Type":"ContainerStarted","Data":"e1d4af594774c3e664e68fc8bc6bfca6706a5e784900ff3cd74fb6dd176e629e"}
Jul 07 11:50:51.838620 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:51.838575    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" podStartSLOduration=1.838559784 podStartE2EDuration="1.838559784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:50:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:50:51.83691399 +0000 UTC m=+2133.375605402" watchObservedRunningTime="2026-07-07 11:50:51.838559784 +0000 UTC m=+2133.377251195"
Jul 07 11:50:52.212708 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:52.212677    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:52.218031 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:52.218006    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:52.820408 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:52.820374    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:52.821306 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:52.821286    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:50:53.802781 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:53.802755    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:50:53.920167 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:53.920134    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:54.037757 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.037722    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:54.094644 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.094543    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"]
Jul 07 11:50:54.095000 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.094984    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="52916e31-5677-4581-9baf-5b5a120c9176" containerName="istio-proxy"
Jul 07 11:50:54.097875 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.097853    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:54.101238 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.100964    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-97eb93-serving-cert\""
Jul 07 11:50:54.110573 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.110550    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"]
Jul 07 11:50:54.130536 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.130503    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:54.187115 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.187086    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd64f6d-0c19-48a6-8d83-3743c0efc734-openshift-service-ca\") pod \"maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") " pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:54.187404 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.187367    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd64f6d-0c19-48a6-8d83-3743c0efc734-maas-api-tls\") pod \"maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") " pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:54.187559 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.187541    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97hsn\" (UniqueName: \"kubernetes.io/projected/edd64f6d-0c19-48a6-8d83-3743c0efc734-kube-api-access-97hsn\") pod \"maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") " pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:54.289282 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.289241    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd64f6d-0c19-48a6-8d83-3743c0efc734-openshift-service-ca\") pod \"maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") " pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:54.289451 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.289307    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd64f6d-0c19-48a6-8d83-3743c0efc734-maas-api-tls\") pod \"maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") " pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:54.289451 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.289342    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97hsn\" (UniqueName: \"kubernetes.io/projected/edd64f6d-0c19-48a6-8d83-3743c0efc734-kube-api-access-97hsn\") pod \"maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") " pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:54.290398 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:54.290371    2582 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-97eb93-serving-cert: secret "maas-api-e2e-shared-b-97eb93-serving-cert" not found
Jul 07 11:50:54.290461 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:50:54.290449    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edd64f6d-0c19-48a6-8d83-3743c0efc734-maas-api-tls podName:edd64f6d-0c19-48a6-8d83-3743c0efc734 nodeName:}" failed. No retries permitted until 2026-07-07 11:50:54.790429877 +0000 UTC m=+2136.329121272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/edd64f6d-0c19-48a6-8d83-3743c0efc734-maas-api-tls") pod "maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n" (UID: "edd64f6d-0c19-48a6-8d83-3743c0efc734") : secret "maas-api-e2e-shared-b-97eb93-serving-cert" not found
Jul 07 11:50:54.290526 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.290501    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd64f6d-0c19-48a6-8d83-3743c0efc734-openshift-service-ca\") pod \"maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") " pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:54.300045 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.300019    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97hsn\" (UniqueName: \"kubernetes.io/projected/edd64f6d-0c19-48a6-8d83-3743c0efc734-kube-api-access-97hsn\") pod \"maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") " pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:54.539462 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.539425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:54.640210 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.640158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:54.739674 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.739645    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:54.795871 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.795770    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd64f6d-0c19-48a6-8d83-3743c0efc734-maas-api-tls\") pod \"maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") " pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:54.798501 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:54.798477    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd64f6d-0c19-48a6-8d83-3743c0efc734-maas-api-tls\") pod \"maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") " pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:55.011003 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:55.010975    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:55.172595 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:55.172564    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"]
Jul 07 11:50:55.175924 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:50:55.175900    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd64f6d_0c19_48a6_8d83_3743c0efc734.slice/crio-7669c75b49e78ba1348505626d9f9b21eee438df933423dac08246726d063340 WatchSource:0}: Error finding container 7669c75b49e78ba1348505626d9f9b21eee438df933423dac08246726d063340: Status 404 returned error can't find the container with id 7669c75b49e78ba1348505626d9f9b21eee438df933423dac08246726d063340
Jul 07 11:50:55.837219 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:55.837168    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n" event={"ID":"edd64f6d-0c19-48a6-8d83-3743c0efc734","Type":"ContainerStarted","Data":"7669c75b49e78ba1348505626d9f9b21eee438df933423dac08246726d063340"}
Jul 07 11:50:56.846700 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:56.846667    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n" event={"ID":"edd64f6d-0c19-48a6-8d83-3743c0efc734","Type":"ContainerStarted","Data":"651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09"}
Jul 07 11:50:56.847173 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:56.846839    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:50:56.867370 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:56.867327    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n" podStartSLOduration=2.064244633 podStartE2EDuration="2.867311299s" totalImagesPullingTime="803.066666ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:50:56.86532909 +0000 UTC m=+2138.404020503" watchObservedRunningTime="2026-07-07 11:50:56.867311299 +0000 UTC m=+2138.406002710"
Jul 07 11:50:57.852441 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:57.852351    2582 generic.go:356] "Generic (PLEG): container finished" podID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerID="2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33" exitCode=0
Jul 07 11:50:57.852441 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:57.852428    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" event={"ID":"e6c61125-4862-4011-99f0-cf8307f6e4a6","Type":"ContainerDied","Data":"2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33"}
Jul 07 11:50:57.852880 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:57.852864    2582 scope.go:117] "RemoveContainer" containerID="2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33"
Jul 07 11:50:57.853816 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:57.853790    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:57.853870 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:57.853855    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:57.853909 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:57.853888    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:50:58.325764 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:58.325730    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:50:58.859368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:58.859337    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" event={"ID":"e6c61125-4862-4011-99f0-cf8307f6e4a6","Type":"ContainerStarted","Data":"58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb"}
Jul 07 11:50:59.101464 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:59.101425    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:50:59.102546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:59.102522    2582 patch_prober.go:28] interesting pod/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 07 11:50:59.102615 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:50:59.102587    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 07 11:51:00.101381 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:00.101345    2582 patch_prober.go:28] interesting pod/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 07 11:51:00.101777 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:00.101398    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 07 11:51:00.769099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:00.769055    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:51:01.101869 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:01.101785    2582 patch_prober.go:28] interesting pod/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 07 11:51:01.101869 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:01.101836    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 07 11:51:02.101546 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:02.101509    2582 patch_prober.go:28] interesting pod/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 07 11:51:02.101721 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:02.101562    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 07 11:51:02.857990 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:02.857960    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:51:03.105677 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:03.105649    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:51:03.106719 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:03.106702    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:51:03.427482 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:03.427450    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:51:07.273182 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:07.273146    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:51:08.804136 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:08.804109    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:51:10.192538 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.192504    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"]
Jul 07 11:51:10.197697 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.197680    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.200376 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.200351    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ab56d6\"/\"openshift-service-ca.crt\""
Jul 07 11:51:10.201439 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.201422    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ab56d6\"/\"kube-root-ca.crt\""
Jul 07 11:51:10.201961 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.201940    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ab56d6\"/\"auth-test-model-ab56d6-kserve-self-signed-certs\""
Jul 07 11:51:10.202052 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.201967    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ab56d6\"/\"default-dockercfg-vzhh7\""
Jul 07 11:51:10.206985 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.206958    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"]
Jul 07 11:51:10.280369 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.280340    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:51:10.343680 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.343642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92ht4\" (UniqueName: \"kubernetes.io/projected/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-kube-api-access-92ht4\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.343866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.343757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tls-certs\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.343866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.343803    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-home\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.343866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.343825    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-dshm\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.343866 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.343851    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-model-cache\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.344054 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.343974    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tmp-dir\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.444698 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.444624    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92ht4\" (UniqueName: \"kubernetes.io/projected/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-kube-api-access-92ht4\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.444698 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.444666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tls-certs\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.444698 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.444686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-home\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.444964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.444706    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-dshm\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.444964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.444731    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-model-cache\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.444964 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.444793    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tmp-dir\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.445299 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.445273    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-home\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.445401 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.445342    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-model-cache\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.445401 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.445389    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tmp-dir\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.446916 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.446875    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-dshm\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.447330 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.447292    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tls-certs\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.453332 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.453310    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92ht4\" (UniqueName: \"kubernetes.io/projected/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-kube-api-access-92ht4\") pod \"auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") " pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.508188 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.508159    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:10.651692 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.651667    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"]
Jul 07 11:51:10.654015 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:51:10.653988    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf4b92e9_9d02_4cb7_845d_c26149eb1ab0.slice/crio-3589c9d168b127040a8ee77df70d16e9c8915578c1274e95ba24ec79613b31a9 WatchSource:0}: Error finding container 3589c9d168b127040a8ee77df70d16e9c8915578c1274e95ba24ec79613b31a9: Status 404 returned error can't find the container with id 3589c9d168b127040a8ee77df70d16e9c8915578c1274e95ba24ec79613b31a9
Jul 07 11:51:10.930313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.930280    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw" event={"ID":"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0","Type":"ContainerStarted","Data":"234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa"}
Jul 07 11:51:10.930478 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.930322    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw" event={"ID":"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0","Type":"ContainerStarted","Data":"3589c9d168b127040a8ee77df70d16e9c8915578c1274e95ba24ec79613b31a9"}
Jul 07 11:51:10.950185 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:10.950088    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw" podStartSLOduration=0.950068873 podStartE2EDuration="950.068873ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:51:10.949138665 +0000 UTC m=+2152.487830092" watchObservedRunningTime="2026-07-07 11:51:10.950068873 +0000 UTC m=+2152.488760286"
Jul 07 11:51:11.936439 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:11.936352    2582 generic.go:356] "Generic (PLEG): container finished" podID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerID="045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8" exitCode=0
Jul 07 11:51:11.936830 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:11.936429    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" event={"ID":"d4f3cabc-017f-4773-be7e-02844a5c9dbb","Type":"ContainerDied","Data":"045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8"}
Jul 07 11:51:11.936830 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:11.936766    2582 scope.go:117] "RemoveContainer" containerID="045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8"
Jul 07 11:51:11.937708 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:11.937679    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:51:11.937779 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:11.937763    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:51:11.937831 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:11.937814    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 07 11:51:12.943257 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:12.943226    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" event={"ID":"d4f3cabc-017f-4773-be7e-02844a5c9dbb","Type":"ContainerStarted","Data":"360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf"}
Jul 07 11:51:13.212405 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:13.212377    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:51:13.212599 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:13.212580    2582 patch_prober.go:28] interesting pod/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 07 11:51:13.212642 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:13.212623    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" podUID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 07 11:51:14.213330 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:14.213294    2582 patch_prober.go:28] interesting pod/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 07 11:51:14.213706 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:14.213361    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" podUID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 07 11:51:14.838645 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:14.838607    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:51:15.212990 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:15.212963    2582 patch_prober.go:28] interesting pod/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 07 11:51:15.213162 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:15.213021    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" podUID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 07 11:51:15.436349 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:15.436317    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:51:15.530418 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:15.530302    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:51:16.212821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:16.212784    2582 patch_prober.go:28] interesting pod/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 07 11:51:16.212996 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:16.212837    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" podUID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 07 11:51:17.216289 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:17.216248    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:51:17.216719 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:17.216532    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:51:17.217179 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:17.217161    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:51:20.508795 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:20.508764    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:20.509335 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:20.509057    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:20.521662 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:20.521641    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:20.994129 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:20.994099    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:51:34.940702 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:34.940668    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:51:38.430657 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:38.430621    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:51:40.184508 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:40.184474    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:51:56.438318 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:51:56.438277    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:52:10.220180 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:52:10.220145    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:52:25.540984 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:52:25.540947    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:52:40.931242 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:52:40.931203    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:52:43.214494 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:52:43.214410    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:52:57.432501 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:52:57.432461    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:53:09.833599 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:09.833560    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:53:22.250780 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:22.250748    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"]
Jul 07 11:53:22.266809 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:22.266778    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:53:23.097955 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:23.097922    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:53:26.125579 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:26.125544    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:53:27.908538 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:27.908503    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"]
Jul 07 11:53:27.911990 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:27.911975    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:27.914626 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:27.914604    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ab56d6\"/\"sub-test-model-ab56d6-kserve-self-signed-certs\""
Jul 07 11:53:27.921435 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:27.921409    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"]
Jul 07 11:53:27.932056 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:27.932029    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:53:28.053255 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.053188    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpgt9\" (UniqueName: \"kubernetes.io/projected/993ac292-09fd-4434-bce3-e48eaef02a9b-kube-api-access-dpgt9\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.053413 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.053264    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-home\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.053413 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.053287    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/993ac292-09fd-4434-bce3-e48eaef02a9b-tls-certs\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.053413 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.053397    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-model-cache\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.053524 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.053435    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-tmp-dir\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.053524 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.053493    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-dshm\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.154308 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.154269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpgt9\" (UniqueName: \"kubernetes.io/projected/993ac292-09fd-4434-bce3-e48eaef02a9b-kube-api-access-dpgt9\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.154308 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.154312    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-home\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.154514 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.154330    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/993ac292-09fd-4434-bce3-e48eaef02a9b-tls-certs\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.154514 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.154368    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-model-cache\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.154514 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.154393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-tmp-dir\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.154644 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.154559    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-dshm\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.154769 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.154746    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-home\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.154848 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.154830    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-model-cache\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.154909 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.154830    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-tmp-dir\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.156703 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.156674    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-dshm\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.156932 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.156913    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/993ac292-09fd-4434-bce3-e48eaef02a9b-tls-certs\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.164027 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.163972    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpgt9\" (UniqueName: \"kubernetes.io/projected/993ac292-09fd-4434-bce3-e48eaef02a9b-kube-api-access-dpgt9\") pod \"sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") " pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.221700 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.221677    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:28.356629 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.356602    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"]
Jul 07 11:53:28.357871 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:53:28.357843    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod993ac292_09fd_4434_bce3_e48eaef02a9b.slice/crio-c323fdfef1a69deb1c271a0201a962988ea0f3946296981a13b3229ef661d89a WatchSource:0}: Error finding container c323fdfef1a69deb1c271a0201a962988ea0f3946296981a13b3229ef661d89a: Status 404 returned error can't find the container with id c323fdfef1a69deb1c271a0201a962988ea0f3946296981a13b3229ef661d89a
Jul 07 11:53:28.613360 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.613319    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9" event={"ID":"993ac292-09fd-4434-bce3-e48eaef02a9b","Type":"ContainerStarted","Data":"653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103"}
Jul 07 11:53:28.613360 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.613365    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9" event={"ID":"993ac292-09fd-4434-bce3-e48eaef02a9b","Type":"ContainerStarted","Data":"c323fdfef1a69deb1c271a0201a962988ea0f3946296981a13b3229ef661d89a"}
Jul 07 11:53:28.646226 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:28.646154    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9" podStartSLOduration=1.646139091 podStartE2EDuration="1.646139091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:53:28.644966843 +0000 UTC m=+2290.183658256" watchObservedRunningTime="2026-07-07 11:53:28.646139091 +0000 UTC m=+2290.184830481"
Jul 07 11:53:37.271314 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.271252    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw" podUID="bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" containerName="main" containerID="cri-o://234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa" gracePeriod=45
Jul 07 11:53:37.527590 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.527535    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:53:37.642914 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.642887    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-model-cache\") pod \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") "
Jul 07 11:53:37.643117 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.642922    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92ht4\" (UniqueName: \"kubernetes.io/projected/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-kube-api-access-92ht4\") pod \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") "
Jul 07 11:53:37.643117 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.642954    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-dshm\") pod \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") "
Jul 07 11:53:37.643117 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.642980    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tls-certs\") pod \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") "
Jul 07 11:53:37.643117 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.643018    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-home\") pod \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") "
Jul 07 11:53:37.643117 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.643062    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tmp-dir\") pod \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\" (UID: \"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0\") "
Jul 07 11:53:37.643452 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.643222    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" (UID: "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:53:37.643452 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.643317    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-home" (OuterVolumeSpecName: "home") pod "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" (UID: "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:53:37.643452 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.643377    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" (UID: "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:53:37.643452 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.643399    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-model-cache\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:53:37.645254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.645224    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-dshm" (OuterVolumeSpecName: "dshm") pod "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" (UID: "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:53:37.645355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.645274    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" (UID: "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:53:37.645397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.645353    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-kube-api-access-92ht4" (OuterVolumeSpecName: "kube-api-access-92ht4") pod "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" (UID: "bf4b92e9-9d02-4cb7-845d-c26149eb1ab0"). InnerVolumeSpecName "kube-api-access-92ht4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:53:37.668933 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.668904    2582 generic.go:356] "Generic (PLEG): container finished" podID="bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" containerID="234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa" exitCode=0
Jul 07 11:53:37.669049 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.668956    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"
Jul 07 11:53:37.669049 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.668992    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw" event={"ID":"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0","Type":"ContainerDied","Data":"234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa"}
Jul 07 11:53:37.669134 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.669049    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw" event={"ID":"bf4b92e9-9d02-4cb7-845d-c26149eb1ab0","Type":"ContainerDied","Data":"3589c9d168b127040a8ee77df70d16e9c8915578c1274e95ba24ec79613b31a9"}
Jul 07 11:53:37.669134 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.669073    2582 scope.go:117] "RemoveContainer" containerID="234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa"
Jul 07 11:53:37.679560 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.679545    2582 scope.go:117] "RemoveContainer" containerID="234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa"
Jul 07 11:53:37.679808 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:53:37.679779    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa\": container with ID starting with 234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa not found: ID does not exist" containerID="234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa"
Jul 07 11:53:37.679868 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.679806    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa"} err="failed to get container status \"234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa\": rpc error: code = NotFound desc = could not find container \"234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa\": container with ID starting with 234d5486d6bb52472cf3dce1397a3bc8374ed944b620e0107db826207b16bbaa not found: ID does not exist"
Jul 07 11:53:37.696945 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.696909    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"]
Jul 07 11:53:37.698451 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.698428    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/auth-test-model-ab56d6-kserve-77d4865c9f-pnlgw"]
Jul 07 11:53:37.743868 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.743843    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92ht4\" (UniqueName: \"kubernetes.io/projected/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-kube-api-access-92ht4\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:53:37.743868 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.743867    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-dshm\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:53:37.743990 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.743877    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tls-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:53:37.743990 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.743885    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-home\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:53:37.743990 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:37.743893    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0-tmp-dir\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:53:38.221977 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:38.221945    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:38.222165 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:38.221992    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:38.234316 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:38.234292    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:38.686743 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:38.686657    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:53:39.025125 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:39.025092    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" path="/var/lib/kubelet/pods/bf4b92e9-9d02-4cb7-845d-c26149eb1ab0/volumes"
Jul 07 11:53:40.324335 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:53:40.324303    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:54:03.424380 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:03.424343    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:54:07.425188 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:07.425153    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:54:20.936953 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:20.936918    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:54:36.700826 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:36.700794    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"]
Jul 07 11:54:36.721900 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:36.721867    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:54:37.636734 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:37.636686    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:54:41.632368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:41.632328    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:54:45.221402 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:45.221371    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:54:45.326025 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:45.325986    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:54:50.628929 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.628889    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"]
Jul 07 11:54:50.629380 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.629300    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" containerName="main"
Jul 07 11:54:50.629380 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.629312    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" containerName="main"
Jul 07 11:54:50.629380 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.629378    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf4b92e9-9d02-4cb7-845d-c26149eb1ab0" containerName="main"
Jul 07 11:54:50.632647 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.632630    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.638044 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.638021    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ab56d6\"/\"rate-test-model-ab56d6-kserve-self-signed-certs\""
Jul 07 11:54:50.648912 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.648892    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"]
Jul 07 11:54:50.677888 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.677864    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-model-cache\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.678003 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.677933    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-dshm\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.678053 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.678000    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9whh\" (UniqueName: \"kubernetes.io/projected/3ee46b75-b4f9-46c8-b068-b7a290caf68e-kube-api-access-z9whh\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.678099 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.678072    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tls-certs\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.678140 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.678113    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tmp-dir\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.678188 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.678147    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-home\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.701072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.701043    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:54:50.779601 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.779579    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-home\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.779719 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.779647    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-model-cache\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.779719 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.779686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-dshm\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.779719 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.779704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9whh\" (UniqueName: \"kubernetes.io/projected/3ee46b75-b4f9-46c8-b068-b7a290caf68e-kube-api-access-z9whh\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.779892 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.779726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tls-certs\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.779892 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.779747    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tmp-dir\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.780051 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.780033    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-model-cache\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.780118 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.780087    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-home\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.780161 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.780128    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tmp-dir\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.782025 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.782007    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-dshm\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.782177 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.782159    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tls-certs\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.789625 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.789601    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9whh\" (UniqueName: \"kubernetes.io/projected/3ee46b75-b4f9-46c8-b068-b7a290caf68e-kube-api-access-z9whh\") pod \"rate-test-model-ab56d6-kserve-674b69df9-nxs7f\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") " pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:50.942040 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:50.941955    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:54:51.076873 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:51.076851    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"]
Jul 07 11:54:51.078159 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:54:51.078130    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ee46b75_b4f9_46c8_b068_b7a290caf68e.slice/crio-7043755d2d8f311bc8ed8b58406f7454ce723770da89c01abebb4ea5281700c0 WatchSource:0}: Error finding container 7043755d2d8f311bc8ed8b58406f7454ce723770da89c01abebb4ea5281700c0: Status 404 returned error can't find the container with id 7043755d2d8f311bc8ed8b58406f7454ce723770da89c01abebb4ea5281700c0
Jul 07 11:54:51.720323 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:51.720255    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9" podUID="993ac292-09fd-4434-bce3-e48eaef02a9b" containerName="main" containerID="cri-o://653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103" gracePeriod=45
Jul 07 11:54:51.979785 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:51.979724    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:54:52.043366 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.043333    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f" event={"ID":"3ee46b75-b4f9-46c8-b068-b7a290caf68e","Type":"ContainerStarted","Data":"6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271"}
Jul 07 11:54:52.043366 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.043371    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f" event={"ID":"3ee46b75-b4f9-46c8-b068-b7a290caf68e","Type":"ContainerStarted","Data":"7043755d2d8f311bc8ed8b58406f7454ce723770da89c01abebb4ea5281700c0"}
Jul 07 11:54:52.044612 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.044588    2582 generic.go:356] "Generic (PLEG): container finished" podID="993ac292-09fd-4434-bce3-e48eaef02a9b" containerID="653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103" exitCode=0
Jul 07 11:54:52.044710 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.044627    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"
Jul 07 11:54:52.044710 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.044677    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9" event={"ID":"993ac292-09fd-4434-bce3-e48eaef02a9b","Type":"ContainerDied","Data":"653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103"}
Jul 07 11:54:52.044787 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.044713    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9" event={"ID":"993ac292-09fd-4434-bce3-e48eaef02a9b","Type":"ContainerDied","Data":"c323fdfef1a69deb1c271a0201a962988ea0f3946296981a13b3229ef661d89a"}
Jul 07 11:54:52.044787 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.044733    2582 scope.go:117] "RemoveContainer" containerID="653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103"
Jul 07 11:54:52.053203 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.053174    2582 scope.go:117] "RemoveContainer" containerID="653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103"
Jul 07 11:54:52.053526 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:54:52.053507    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103\": container with ID starting with 653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103 not found: ID does not exist" containerID="653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103"
Jul 07 11:54:52.053579 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.053534    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103"} err="failed to get container status \"653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103\": rpc error: code = NotFound desc = could not find container \"653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103\": container with ID starting with 653a8fc9c13f29075e172bd44e5bc096e3a296c82673989cd5f65565a864b103 not found: ID does not exist"
Jul 07 11:54:52.064361 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.064320    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f" podStartSLOduration=2.064301314 podStartE2EDuration="2.064301314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:54:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:54:52.062248988 +0000 UTC m=+2373.600940391" watchObservedRunningTime="2026-07-07 11:54:52.064301314 +0000 UTC m=+2373.602992726"
Jul 07 11:54:52.093986 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.093962    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-dshm\") pod \"993ac292-09fd-4434-bce3-e48eaef02a9b\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") "
Jul 07 11:54:52.094096 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.093992    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpgt9\" (UniqueName: \"kubernetes.io/projected/993ac292-09fd-4434-bce3-e48eaef02a9b-kube-api-access-dpgt9\") pod \"993ac292-09fd-4434-bce3-e48eaef02a9b\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") "
Jul 07 11:54:52.094096 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.094007    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-home\") pod \"993ac292-09fd-4434-bce3-e48eaef02a9b\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") "
Jul 07 11:54:52.094096 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.094044    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/993ac292-09fd-4434-bce3-e48eaef02a9b-tls-certs\") pod \"993ac292-09fd-4434-bce3-e48eaef02a9b\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") "
Jul 07 11:54:52.094096 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.094069    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-model-cache\") pod \"993ac292-09fd-4434-bce3-e48eaef02a9b\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") "
Jul 07 11:54:52.094354 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.094112    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-tmp-dir\") pod \"993ac292-09fd-4434-bce3-e48eaef02a9b\" (UID: \"993ac292-09fd-4434-bce3-e48eaef02a9b\") "
Jul 07 11:54:52.094416 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.094396    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-model-cache" (OuterVolumeSpecName: "model-cache") pod "993ac292-09fd-4434-bce3-e48eaef02a9b" (UID: "993ac292-09fd-4434-bce3-e48eaef02a9b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:54:52.094473 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.094411    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "993ac292-09fd-4434-bce3-e48eaef02a9b" (UID: "993ac292-09fd-4434-bce3-e48eaef02a9b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:54:52.094563 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.094350    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-home" (OuterVolumeSpecName: "home") pod "993ac292-09fd-4434-bce3-e48eaef02a9b" (UID: "993ac292-09fd-4434-bce3-e48eaef02a9b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:54:52.094655 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.094598    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-model-cache\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:54:52.094749 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.094737    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-tmp-dir\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:54:52.096299 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.096271    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/993ac292-09fd-4434-bce3-e48eaef02a9b-kube-api-access-dpgt9" (OuterVolumeSpecName: "kube-api-access-dpgt9") pod "993ac292-09fd-4434-bce3-e48eaef02a9b" (UID: "993ac292-09fd-4434-bce3-e48eaef02a9b"). InnerVolumeSpecName "kube-api-access-dpgt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:54:52.096564 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.096545    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/993ac292-09fd-4434-bce3-e48eaef02a9b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "993ac292-09fd-4434-bce3-e48eaef02a9b" (UID: "993ac292-09fd-4434-bce3-e48eaef02a9b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:54:52.096634 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.096544    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-dshm" (OuterVolumeSpecName: "dshm") pod "993ac292-09fd-4434-bce3-e48eaef02a9b" (UID: "993ac292-09fd-4434-bce3-e48eaef02a9b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:54:52.196023 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.195996    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-dshm\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:54:52.196023 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.196021    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpgt9\" (UniqueName: \"kubernetes.io/projected/993ac292-09fd-4434-bce3-e48eaef02a9b-kube-api-access-dpgt9\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:54:52.196023 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.196031    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/993ac292-09fd-4434-bce3-e48eaef02a9b-home\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:54:52.196285 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.196039    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/993ac292-09fd-4434-bce3-e48eaef02a9b-tls-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:54:52.369857 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.369826    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"]
Jul 07 11:54:52.371756 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:52.371734    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/sub-test-model-ab56d6-kserve-6f7c5f487f-2qxt9"]
Jul 07 11:54:53.025714 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:53.025677    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="993ac292-09fd-4434-bce3-e48eaef02a9b" path="/var/lib/kubelet/pods/993ac292-09fd-4434-bce3-e48eaef02a9b/volumes"
Jul 07 11:54:54.726540 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:54:54.726509    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:55:00.942359 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:00.942328    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:55:00.942736 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:00.942381    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:55:00.954667 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:00.954641    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:55:01.100863 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:01.100834    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:55:15.633384 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:15.633349    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:55:19.092117 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:19.092090    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:55:19.097822 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:19.097791    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:55:27.145651 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:27.145620    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:55:29.984640 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:29.984602    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:55:30.083314 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:30.083273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:55:30.928525 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:30.928489    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:55:57.619990 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:55:57.619912    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:56:02.110039 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:02.110001    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:07.978970 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:07.978937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:08.081089 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:08.081054    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:10.625946 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:10.625907    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:56:16.151295 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:16.151259    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:16.951610 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:16.951579    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"]
Jul 07 11:56:17.361868 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:17.361833    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:17.820394 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:17.820359    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:19.037476 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.037446    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:19.400857 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.400765    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:19.500550 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.500511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:19.600693 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.600612    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:19.691306 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.691231    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"]
Jul 07 11:56:19.691500 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.691475    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899" podUID="b4a12624-1229-426a-8b93-08fa98d4be01" containerName="maas-api" containerID="cri-o://cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24" gracePeriod=30
Jul 07 11:56:19.715405 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.715376    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:19.939291 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.939270    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:56:19.974452 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.974427    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a12624-1229-426a-8b93-08fa98d4be01-openshift-service-ca\") pod \"b4a12624-1229-426a-8b93-08fa98d4be01\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") "
Jul 07 11:56:19.974575 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.974551    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a12624-1229-426a-8b93-08fa98d4be01-maas-api-tls\") pod \"b4a12624-1229-426a-8b93-08fa98d4be01\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") "
Jul 07 11:56:19.974626 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.974609    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9fn5\" (UniqueName: \"kubernetes.io/projected/b4a12624-1229-426a-8b93-08fa98d4be01-kube-api-access-g9fn5\") pod \"b4a12624-1229-426a-8b93-08fa98d4be01\" (UID: \"b4a12624-1229-426a-8b93-08fa98d4be01\") "
Jul 07 11:56:19.974821 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.974801    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4a12624-1229-426a-8b93-08fa98d4be01-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4a12624-1229-426a-8b93-08fa98d4be01" (UID: "b4a12624-1229-426a-8b93-08fa98d4be01"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:56:19.974912 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.974902    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a12624-1229-426a-8b93-08fa98d4be01-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:56:19.976520 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.976492    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4a12624-1229-426a-8b93-08fa98d4be01-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4a12624-1229-426a-8b93-08fa98d4be01" (UID: "b4a12624-1229-426a-8b93-08fa98d4be01"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:56:19.976626 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:19.976608    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a12624-1229-426a-8b93-08fa98d4be01-kube-api-access-g9fn5" (OuterVolumeSpecName: "kube-api-access-g9fn5") pod "b4a12624-1229-426a-8b93-08fa98d4be01" (UID: "b4a12624-1229-426a-8b93-08fa98d4be01"). InnerVolumeSpecName "kube-api-access-g9fn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:56:20.076082 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.076053    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4a12624-1229-426a-8b93-08fa98d4be01-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:56:20.076082 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.076079    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9fn5\" (UniqueName: \"kubernetes.io/projected/b4a12624-1229-426a-8b93-08fa98d4be01-kube-api-access-g9fn5\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:56:20.474891 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.474860    2582 generic.go:356] "Generic (PLEG): container finished" podID="b4a12624-1229-426a-8b93-08fa98d4be01" containerID="cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24" exitCode=0
Jul 07 11:56:20.475048 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.474898    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899" event={"ID":"b4a12624-1229-426a-8b93-08fa98d4be01","Type":"ContainerDied","Data":"cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24"}
Jul 07 11:56:20.475048 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.474908    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"
Jul 07 11:56:20.475048 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.474941    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899" event={"ID":"b4a12624-1229-426a-8b93-08fa98d4be01","Type":"ContainerDied","Data":"4e5a7ad2c343174581f411acacd5ac938cfe5aff3742c035cdac568e4143d96b"}
Jul 07 11:56:20.475048 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.474961    2582 scope.go:117] "RemoveContainer" containerID="cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24"
Jul 07 11:56:20.484721 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.484704    2582 scope.go:117] "RemoveContainer" containerID="cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24"
Jul 07 11:56:20.484958 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:56:20.484939    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24\": container with ID starting with cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24 not found: ID does not exist" containerID="cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24"
Jul 07 11:56:20.485004 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.484969    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24"} err="failed to get container status \"cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24\": rpc error: code = NotFound desc = could not find container \"cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24\": container with ID starting with cd8048cf76ed1dee501003074a5e899d5f08fa480d8f9d3040690cbeb29f9a24 not found: ID does not exist"
Jul 07 11:56:20.496972 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.496953    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"]
Jul 07 11:56:20.501288 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:20.501265    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ab56d6-6f4dd78b45-z4899"]
Jul 07 11:56:21.025679 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:21.025646    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4a12624-1229-426a-8b93-08fa98d4be01" path="/var/lib/kubelet/pods/b4a12624-1229-426a-8b93-08fa98d4be01/volumes"
Jul 07 11:56:23.425528 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:23.425495    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:56:24.691102 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:24.691059    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:56:31.972011 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:31.971976    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f" podUID="3ee46b75-b4f9-46c8-b068-b7a290caf68e" containerName="main" containerID="cri-o://6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271" gracePeriod=45
Jul 07 11:56:32.223481 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.223426    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:56:32.280086 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280060    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tls-certs\") pod \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") "
Jul 07 11:56:32.280232 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280150    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-model-cache\") pod \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") "
Jul 07 11:56:32.280232 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280169    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-dshm\") pod \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") "
Jul 07 11:56:32.280232 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280226    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tmp-dir\") pod \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") "
Jul 07 11:56:32.280396 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280244    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9whh\" (UniqueName: \"kubernetes.io/projected/3ee46b75-b4f9-46c8-b068-b7a290caf68e-kube-api-access-z9whh\") pod \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") "
Jul 07 11:56:32.280396 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280272    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-home\") pod \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\" (UID: \"3ee46b75-b4f9-46c8-b068-b7a290caf68e\") "
Jul 07 11:56:32.280504 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280465    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3ee46b75-b4f9-46c8-b068-b7a290caf68e" (UID: "3ee46b75-b4f9-46c8-b068-b7a290caf68e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:32.280504 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280477    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-model-cache" (OuterVolumeSpecName: "model-cache") pod "3ee46b75-b4f9-46c8-b068-b7a290caf68e" (UID: "3ee46b75-b4f9-46c8-b068-b7a290caf68e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:32.280620 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280575    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-home" (OuterVolumeSpecName: "home") pod "3ee46b75-b4f9-46c8-b068-b7a290caf68e" (UID: "3ee46b75-b4f9-46c8-b068-b7a290caf68e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:32.280620 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280610    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-model-cache\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.280701 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.280627    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tmp-dir\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.282277 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.282254    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-dshm" (OuterVolumeSpecName: "dshm") pod "3ee46b75-b4f9-46c8-b068-b7a290caf68e" (UID: "3ee46b75-b4f9-46c8-b068-b7a290caf68e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:32.282381 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.282311    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3ee46b75-b4f9-46c8-b068-b7a290caf68e" (UID: "3ee46b75-b4f9-46c8-b068-b7a290caf68e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:56:32.282458 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.282442    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ee46b75-b4f9-46c8-b068-b7a290caf68e-kube-api-access-z9whh" (OuterVolumeSpecName: "kube-api-access-z9whh") pod "3ee46b75-b4f9-46c8-b068-b7a290caf68e" (UID: "3ee46b75-b4f9-46c8-b068-b7a290caf68e"). InnerVolumeSpecName "kube-api-access-z9whh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:56:32.381857 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.381831    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-dshm\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.381857 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.381857    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9whh\" (UniqueName: \"kubernetes.io/projected/3ee46b75-b4f9-46c8-b068-b7a290caf68e-kube-api-access-z9whh\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.382008 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.381869    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ee46b75-b4f9-46c8-b068-b7a290caf68e-home\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.382008 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.381877    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ee46b75-b4f9-46c8-b068-b7a290caf68e-tls-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.534237 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.534126    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ee46b75-b4f9-46c8-b068-b7a290caf68e" containerID="6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271" exitCode=0
Jul 07 11:56:32.534237 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.534164    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"
Jul 07 11:56:32.534416 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.534237    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f" event={"ID":"3ee46b75-b4f9-46c8-b068-b7a290caf68e","Type":"ContainerDied","Data":"6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271"}
Jul 07 11:56:32.534416 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.534291    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f" event={"ID":"3ee46b75-b4f9-46c8-b068-b7a290caf68e","Type":"ContainerDied","Data":"7043755d2d8f311bc8ed8b58406f7454ce723770da89c01abebb4ea5281700c0"}
Jul 07 11:56:32.534416 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.534314    2582 scope.go:117] "RemoveContainer" containerID="6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271"
Jul 07 11:56:32.543933 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.543914    2582 scope.go:117] "RemoveContainer" containerID="6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271"
Jul 07 11:56:32.544188 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:56:32.544163    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271\": container with ID starting with 6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271 not found: ID does not exist" containerID="6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271"
Jul 07 11:56:32.544318 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.544209    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271"} err="failed to get container status \"6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271\": rpc error: code = NotFound desc = could not find container \"6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271\": container with ID starting with 6b474d890855a61f0f7b9f2672c1a61594bcdfbb6ad69b92d6a6b0039b3ea271 not found: ID does not exist"
Jul 07 11:56:32.559244 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.558966    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"]
Jul 07 11:56:32.563416 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:32.563390    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ab56d6/rate-test-model-ab56d6-kserve-674b69df9-nxs7f"]
Jul 07 11:56:33.026480 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:56:33.026448    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ee46b75-b4f9-46c8-b068-b7a290caf68e" path="/var/lib/kubelet/pods/3ee46b75-b4f9-46c8-b068-b7a290caf68e/volumes"
Jul 07 11:57:01.780819 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:01.780781    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:02.086942 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:02.086860    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"]
Jul 07 11:57:02.087147 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:02.087122    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy" containerID="cri-o://58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb" gracePeriod=30
Jul 07 11:57:02.131946 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:02.131912    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:02.340035 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:02.339952    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:02.513189 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:02.513161    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:02.913916 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:02.913875    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:03.020937 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.020903    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:03.108071 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.108039    2582 patch_prober.go:28] interesting pod/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 11:57:03.108246 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.108094    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 11:57:03.154455 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.154414    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:03.337884 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.337855    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"]
Jul 07 11:57:03.338064 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.338040    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n" podUID="edd64f6d-0c19-48a6-8d83-3743c0efc734" containerName="maas-api" containerID="cri-o://651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09" gracePeriod=30
Jul 07 11:57:03.399279 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.399248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:03.584127 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.584100    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:57:03.654824 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.654761    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd64f6d-0c19-48a6-8d83-3743c0efc734-maas-api-tls\") pod \"edd64f6d-0c19-48a6-8d83-3743c0efc734\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") "
Jul 07 11:57:03.654941 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.654851    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97hsn\" (UniqueName: \"kubernetes.io/projected/edd64f6d-0c19-48a6-8d83-3743c0efc734-kube-api-access-97hsn\") pod \"edd64f6d-0c19-48a6-8d83-3743c0efc734\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") "
Jul 07 11:57:03.654941 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.654872    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd64f6d-0c19-48a6-8d83-3743c0efc734-openshift-service-ca\") pod \"edd64f6d-0c19-48a6-8d83-3743c0efc734\" (UID: \"edd64f6d-0c19-48a6-8d83-3743c0efc734\") "
Jul 07 11:57:03.655291 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.655261    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edd64f6d-0c19-48a6-8d83-3743c0efc734-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "edd64f6d-0c19-48a6-8d83-3743c0efc734" (UID: "edd64f6d-0c19-48a6-8d83-3743c0efc734"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:57:03.656876 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.656853    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edd64f6d-0c19-48a6-8d83-3743c0efc734-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "edd64f6d-0c19-48a6-8d83-3743c0efc734" (UID: "edd64f6d-0c19-48a6-8d83-3743c0efc734"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:57:03.656988 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.656893    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd64f6d-0c19-48a6-8d83-3743c0efc734-kube-api-access-97hsn" (OuterVolumeSpecName: "kube-api-access-97hsn") pod "edd64f6d-0c19-48a6-8d83-3743c0efc734" (UID: "edd64f6d-0c19-48a6-8d83-3743c0efc734"). InnerVolumeSpecName "kube-api-access-97hsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:57:03.685853 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.685831    2582 generic.go:356] "Generic (PLEG): container finished" podID="edd64f6d-0c19-48a6-8d83-3743c0efc734" containerID="651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09" exitCode=0
Jul 07 11:57:03.685945 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.685870    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"
Jul 07 11:57:03.685945 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.685905    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n" event={"ID":"edd64f6d-0c19-48a6-8d83-3743c0efc734","Type":"ContainerDied","Data":"651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09"}
Jul 07 11:57:03.686026 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.685951    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n" event={"ID":"edd64f6d-0c19-48a6-8d83-3743c0efc734","Type":"ContainerDied","Data":"7669c75b49e78ba1348505626d9f9b21eee438df933423dac08246726d063340"}
Jul 07 11:57:03.686026 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.685973    2582 scope.go:117] "RemoveContainer" containerID="651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09"
Jul 07 11:57:03.695594 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.695575    2582 scope.go:117] "RemoveContainer" containerID="651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09"
Jul 07 11:57:03.695856 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:57:03.695839    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09\": container with ID starting with 651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09 not found: ID does not exist" containerID="651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09"
Jul 07 11:57:03.695925 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.695870    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09"} err="failed to get container status \"651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09\": rpc error: code = NotFound desc = could not find container \"651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09\": container with ID starting with 651ef2ce2ac0a484f6b0f9aadeabdf70c5dc0ee5dfac2b5138f8f8e1a7a64f09 not found: ID does not exist"
Jul 07 11:57:03.708654 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.708603    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"]
Jul 07 11:57:03.710537 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.710514    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97eb93-fd68fcf6c-5kx4n"]
Jul 07 11:57:03.756130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.756104    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97hsn\" (UniqueName: \"kubernetes.io/projected/edd64f6d-0c19-48a6-8d83-3743c0efc734-kube-api-access-97hsn\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:03.756130 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.756127    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd64f6d-0c19-48a6-8d83-3743c0efc734-openshift-service-ca\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:03.756283 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:03.756142    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd64f6d-0c19-48a6-8d83-3743c0efc734-maas-api-tls\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:05.024885 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:05.024856    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edd64f6d-0c19-48a6-8d83-3743c0efc734" path="/var/lib/kubelet/pods/edd64f6d-0c19-48a6-8d83-3743c0efc734/volumes"
Jul 07 11:57:06.028335 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:06.028309    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2lhcn"]
Jul 07 11:57:07.343542 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.343520    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:57:07.386618 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.386535    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-podinfo\") pod \"e6c61125-4862-4011-99f0-cf8307f6e4a6\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") "
Jul 07 11:57:07.386618 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.386582    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-socket\") pod \"e6c61125-4862-4011-99f0-cf8307f6e4a6\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") "
Jul 07 11:57:07.386805 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.386624    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-data\") pod \"e6c61125-4862-4011-99f0-cf8307f6e4a6\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") "
Jul 07 11:57:07.386805 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.386657    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-envoy\") pod \"e6c61125-4862-4011-99f0-cf8307f6e4a6\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") "
Jul 07 11:57:07.386805 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.386754    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-token\") pod \"e6c61125-4862-4011-99f0-cf8307f6e4a6\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") "
Jul 07 11:57:07.386946 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.386814    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-certs\") pod \"e6c61125-4862-4011-99f0-cf8307f6e4a6\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") "
Jul 07 11:57:07.386946 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.386809    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e6c61125-4862-4011-99f0-cf8307f6e4a6" (UID: "e6c61125-4862-4011-99f0-cf8307f6e4a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:57:07.386946 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.386866    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-credential-socket\") pod \"e6c61125-4862-4011-99f0-cf8307f6e4a6\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") "
Jul 07 11:57:07.386946 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.386893    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6c61125-4862-4011-99f0-cf8307f6e4a6-istiod-ca-cert\") pod \"e6c61125-4862-4011-99f0-cf8307f6e4a6\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") "
Jul 07 11:57:07.386946 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.386918    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjv6f\" (UniqueName: \"kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-kube-api-access-jjv6f\") pod \"e6c61125-4862-4011-99f0-cf8307f6e4a6\" (UID: \"e6c61125-4862-4011-99f0-cf8307f6e4a6\") "
Jul 07 11:57:07.387229 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.387064    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e6c61125-4862-4011-99f0-cf8307f6e4a6" (UID: "e6c61125-4862-4011-99f0-cf8307f6e4a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:57:07.387313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.387259    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:07.387313 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.387277    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-workload-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:07.387511 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.387411    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6c61125-4862-4011-99f0-cf8307f6e4a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e6c61125-4862-4011-99f0-cf8307f6e4a6" (UID: "e6c61125-4862-4011-99f0-cf8307f6e4a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:57:07.387511 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.387483    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e6c61125-4862-4011-99f0-cf8307f6e4a6" (UID: "e6c61125-4862-4011-99f0-cf8307f6e4a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:57:07.387696 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.387624    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "e6c61125-4862-4011-99f0-cf8307f6e4a6" (UID: "e6c61125-4862-4011-99f0-cf8307f6e4a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:57:07.389124 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.389099    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-kube-api-access-jjv6f" (OuterVolumeSpecName: "kube-api-access-jjv6f") pod "e6c61125-4862-4011-99f0-cf8307f6e4a6" (UID: "e6c61125-4862-4011-99f0-cf8307f6e4a6"). InnerVolumeSpecName "kube-api-access-jjv6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:57:07.389336 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.389123    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "e6c61125-4862-4011-99f0-cf8307f6e4a6" (UID: "e6c61125-4862-4011-99f0-cf8307f6e4a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:57:07.389336 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.389329    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e6c61125-4862-4011-99f0-cf8307f6e4a6" (UID: "e6c61125-4862-4011-99f0-cf8307f6e4a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:57:07.389479 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.389347    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e6c61125-4862-4011-99f0-cf8307f6e4a6" (UID: "e6c61125-4862-4011-99f0-cf8307f6e4a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:57:07.487843 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.487815    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-token\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:07.487843 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.487842    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-credential-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:07.488005 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.487852    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6c61125-4862-4011-99f0-cf8307f6e4a6-istiod-ca-cert\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:07.488005 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.487861    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjv6f\" (UniqueName: \"kubernetes.io/projected/e6c61125-4862-4011-99f0-cf8307f6e4a6-kube-api-access-jjv6f\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:07.488005 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.487870    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-podinfo\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:07.488005 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.487878    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-data\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:07.488005 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.487888    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6c61125-4862-4011-99f0-cf8307f6e4a6-istio-envoy\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:07.708651 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.708564    2582 generic.go:356] "Generic (PLEG): container finished" podID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerID="58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb" exitCode=0
Jul 07 11:57:07.708651 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.708610    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"
Jul 07 11:57:07.708826 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.708655    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" event={"ID":"e6c61125-4862-4011-99f0-cf8307f6e4a6","Type":"ContainerDied","Data":"58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb"}
Jul 07 11:57:07.708826 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.708709    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c" event={"ID":"e6c61125-4862-4011-99f0-cf8307f6e4a6","Type":"ContainerDied","Data":"0285fe844ad9a658d3061dce50515676e19bb1b5464aaf2c0e994459f2284f1b"}
Jul 07 11:57:07.708826 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.708731    2582 scope.go:117] "RemoveContainer" containerID="58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb"
Jul 07 11:57:07.717737 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.717722    2582 scope.go:117] "RemoveContainer" containerID="2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33"
Jul 07 11:57:07.725324 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.725309    2582 scope.go:117] "RemoveContainer" containerID="58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb"
Jul 07 11:57:07.725544 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:57:07.725526    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb\": container with ID starting with 58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb not found: ID does not exist" containerID="58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb"
Jul 07 11:57:07.725598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.725551    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb"} err="failed to get container status \"58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb\": rpc error: code = NotFound desc = could not find container \"58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb\": container with ID starting with 58d490db9c1575f072a5cd1ec05eb8259d9dd976eaee8db467147cbba519efeb not found: ID does not exist"
Jul 07 11:57:07.725598 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.725567    2582 scope.go:117] "RemoveContainer" containerID="2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33"
Jul 07 11:57:07.725782 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:57:07.725766    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33\": container with ID starting with 2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33 not found: ID does not exist" containerID="2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33"
Jul 07 11:57:07.725826 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.725785    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33"} err="failed to get container status \"2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33\": rpc error: code = NotFound desc = could not find container \"2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33\": container with ID starting with 2e04a209cd106badc989007b0ae1bd09f8f6a1f18be40145f4e2f2c0435f3e33 not found: ID does not exist"
Jul 07 11:57:07.731031 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.731012    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"]
Jul 07 11:57:07.735332 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:07.735312    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-ab56d6-openshift-default-6c8f6c687-5gh4c"]
Jul 07 11:57:08.805117 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:08.805089    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:08.907386 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:08.907344    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:09.024996 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:09.024963    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" path="/var/lib/kubelet/pods/e6c61125-4862-4011-99f0-cf8307f6e4a6/volumes"
Jul 07 11:57:09.064528 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:09.064443    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"]
Jul 07 11:57:09.064755 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:09.064728    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" podUID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerName="istio-proxy" containerID="cri-o://360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf" gracePeriod=30
Jul 07 11:57:09.080584 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:09.080556    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:09.326629 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:09.326552    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:14.319713 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.319691    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:57:14.442268 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442146    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-data\") pod \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") "
Jul 07 11:57:14.442268 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442240    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-certs\") pod \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") "
Jul 07 11:57:14.442497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442286    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gnjk\" (UniqueName: \"kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-kube-api-access-8gnjk\") pod \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") "
Jul 07 11:57:14.442497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442318    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istiod-ca-cert\") pod \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") "
Jul 07 11:57:14.442497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442340    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-credential-socket\") pod \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") "
Jul 07 11:57:14.442497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442372    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-token\") pod \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") "
Jul 07 11:57:14.442497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442405    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-podinfo\") pod \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") "
Jul 07 11:57:14.442497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442428    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-socket\") pod \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") "
Jul 07 11:57:14.442497 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442475    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-envoy\") pod \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\" (UID: \"d4f3cabc-017f-4773-be7e-02844a5c9dbb\") "
Jul 07 11:57:14.442822 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442517    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4f3cabc-017f-4773-be7e-02844a5c9dbb" (UID: "d4f3cabc-017f-4773-be7e-02844a5c9dbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:57:14.442822 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442759    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-certs\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:14.442925 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442853    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4f3cabc-017f-4773-be7e-02844a5c9dbb" (UID: "d4f3cabc-017f-4773-be7e-02844a5c9dbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:57:14.442925 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442898    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4f3cabc-017f-4773-be7e-02844a5c9dbb" (UID: "d4f3cabc-017f-4773-be7e-02844a5c9dbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:57:14.442925 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.442910    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4f3cabc-017f-4773-be7e-02844a5c9dbb" (UID: "d4f3cabc-017f-4773-be7e-02844a5c9dbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:57:14.443080 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.443038    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4f3cabc-017f-4773-be7e-02844a5c9dbb" (UID: "d4f3cabc-017f-4773-be7e-02844a5c9dbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:57:14.444738 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.444708    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4f3cabc-017f-4773-be7e-02844a5c9dbb" (UID: "d4f3cabc-017f-4773-be7e-02844a5c9dbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:57:14.444857 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.444761    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-kube-api-access-8gnjk" (OuterVolumeSpecName: "kube-api-access-8gnjk") pod "d4f3cabc-017f-4773-be7e-02844a5c9dbb" (UID: "d4f3cabc-017f-4773-be7e-02844a5c9dbb"). InnerVolumeSpecName "kube-api-access-8gnjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:57:14.444976 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.444959    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4f3cabc-017f-4773-be7e-02844a5c9dbb" (UID: "d4f3cabc-017f-4773-be7e-02844a5c9dbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:57:14.445032 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.444991    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4f3cabc-017f-4773-be7e-02844a5c9dbb" (UID: "d4f3cabc-017f-4773-be7e-02844a5c9dbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:57:14.543844 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.543809    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istiod-ca-cert\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:14.543844 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.543836    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-credential-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:14.543844 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.543850    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-token\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:14.544074 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.543861    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-podinfo\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:14.544074 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.543872    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-workload-socket\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:14.544074 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.543883    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-envoy\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:14.544074 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.543895    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f3cabc-017f-4773-be7e-02844a5c9dbb-istio-data\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:14.544074 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.543905    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gnjk\" (UniqueName: \"kubernetes.io/projected/d4f3cabc-017f-4773-be7e-02844a5c9dbb-kube-api-access-8gnjk\") on node \"ip-10-0-133-180.ec2.internal\" DevicePath \"\""
Jul 07 11:57:14.727884 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.727854    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723730-g5qgj_2bc5f507-00fe-4de9-8c24-f59641cdf494/cleanup/0.log"
Jul 07 11:57:14.746225 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.746182    2582 generic.go:356] "Generic (PLEG): container finished" podID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerID="360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf" exitCode=0
Jul 07 11:57:14.746355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.746233    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" event={"ID":"d4f3cabc-017f-4773-be7e-02844a5c9dbb","Type":"ContainerDied","Data":"360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf"}
Jul 07 11:57:14.746355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.746259    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"
Jul 07 11:57:14.746355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.746289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr" event={"ID":"d4f3cabc-017f-4773-be7e-02844a5c9dbb","Type":"ContainerDied","Data":"e1d4af594774c3e664e68fc8bc6bfca6706a5e784900ff3cd74fb6dd176e629e"}
Jul 07 11:57:14.746355 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.746312    2582 scope.go:117] "RemoveContainer" containerID="360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf"
Jul 07 11:57:14.755610 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.755594    2582 scope.go:117] "RemoveContainer" containerID="045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8"
Jul 07 11:57:14.763505 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.763484    2582 scope.go:117] "RemoveContainer" containerID="360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf"
Jul 07 11:57:14.763764 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:57:14.763742    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf\": container with ID starting with 360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf not found: ID does not exist" containerID="360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf"
Jul 07 11:57:14.763819 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.763776    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf"} err="failed to get container status \"360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf\": rpc error: code = NotFound desc = could not find container \"360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf\": container with ID starting with 360dad4584ff71b1645b00f1f7ca02a078eea825acade4bf70c92faecf507fdf not found: ID does not exist"
Jul 07 11:57:14.763819 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.763801    2582 scope.go:117] "RemoveContainer" containerID="045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8"
Jul 07 11:57:14.764039 ip-10-0-133-180 kubenswrapper[2582]: E0707 11:57:14.764021    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8\": container with ID starting with 045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8 not found: ID does not exist" containerID="045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8"
Jul 07 11:57:14.764093 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.764043    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8"} err="failed to get container status \"045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8\": rpc error: code = NotFound desc = could not find container \"045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8\": container with ID starting with 045a55af46bc4cf0bd37c96660838bc9efec3c81d75196deb4bcaf2f71014ee8 not found: ID does not exist"
Jul 07 11:57:14.768004 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.767979    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"]
Jul 07 11:57:14.774779 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.774759    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-97eb93-openshift-default-5cc9956999-ddjdr"]
Jul 07 11:57:14.839090 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:14.839071    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723745-lclc5_0df681bd-2339-4c56-92a0-7d29c68724fd/cleanup/0.log"
Jul 07 11:57:15.024873 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:15.024794    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" path="/var/lib/kubelet/pods/d4f3cabc-017f-4773-be7e-02844a5c9dbb/volumes"
Jul 07 11:57:15.305481 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:15.305403    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-4kn54_5b2d0fdb-d643-45d2-99bd-851a456b14fa/manager/0.log"
Jul 07 11:57:15.312474 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:15.312455    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-4kn54_5b2d0fdb-d643-45d2-99bd-851a456b14fa/copy-manifests/0.log"
Jul 07 11:57:17.285152 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:17.285123    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-crjgh_12774182-f4f5-46ca-93cb-96f04a2118e8/manager/0.log"
Jul 07 11:57:17.394979 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:17.394955    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6tpdf_cba36d43-9721-40af-a1f1-73793d352705/kuadrant-console-plugin/0.log"
Jul 07 11:57:17.643984 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:17.643910    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7rqcr_31665f10-91e9-4f6a-a2f6-1f87e8389d48/manager/0.log"
Jul 07 11:57:17.753282 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:17.753255    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2lhcn_f7bfbf03-4f84-4b4a-9009-b8f62a653b65/limitador/0.log"
Jul 07 11:57:19.052889 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:19.052843    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c5dbcc7d5-crvzt_1de016bd-1463-4d64-ad1b-09e17dce5d18/router/0.log"
Jul 07 11:57:19.603383 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:19.603348    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-d8wfv_9ca7d180-7e34-455b-940c-6a611f6a7d66/main/0.log"
Jul 07 11:57:24.197630 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:24.197597    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:24.295418 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:24.295385    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log" limit=1024
Jul 07 11:57:27.044125 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:27.044093    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cb8jb_96412d1b-55ef-4558-88c3-4b83911151b5/global-pull-secret-syncer/0.log"
Jul 07 11:57:27.100215 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:27.100168    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cq9jg_9a103de2-c03c-4ba3-9c40-184af233d72a/konnectivity-agent/0.log"
Jul 07 11:57:27.244080 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:27.244043    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-180.ec2.internal_994634ad57956f8c96f482cc22f50bb3/haproxy/0.log"
Jul 07 11:57:32.723544 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:32.723504    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-crjgh_12774182-f4f5-46ca-93cb-96f04a2118e8/manager/0.log"
Jul 07 11:57:32.753072 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:32.753048    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6tpdf_cba36d43-9721-40af-a1f1-73793d352705/kuadrant-console-plugin/0.log"
Jul 07 11:57:32.973752 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:32.973668    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7rqcr_31665f10-91e9-4f6a-a2f6-1f87e8389d48/manager/0.log"
Jul 07 11:57:32.996136 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:32.996101    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2lhcn_f7bfbf03-4f84-4b4a-9009-b8f62a653b65/limitador/0.log"
Jul 07 11:57:34.921570 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:34.921537    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-k46bt_1dbeb02c-2837-40aa-bdc1-e6f3a03a4c46/cluster-monitoring-operator/0.log"
Jul 07 11:57:35.270825 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:35.270790    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mklsj_b284ab20-3f75-40b8-95a2-ddcebd7849c8/node-exporter/0.log"
Jul 07 11:57:35.295014 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:35.294994    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mklsj_b284ab20-3f75-40b8-95a2-ddcebd7849c8/kube-rbac-proxy/0.log"
Jul 07 11:57:35.325488 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:35.325470    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mklsj_b284ab20-3f75-40b8-95a2-ddcebd7849c8/init-textfile/0.log"
Jul 07 11:57:37.089246 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.089215    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-wshqb_9b40451d-1272-4a5d-a8be-fa8a20b6b565/networking-console-plugin/0.log"
Jul 07 11:57:37.413865 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.413769    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"]
Jul 07 11:57:37.414254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414234    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy"
Jul 07 11:57:37.414254 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414253    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy"
Jul 07 11:57:37.414389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414263    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="993ac292-09fd-4434-bce3-e48eaef02a9b" containerName="main"
Jul 07 11:57:37.414389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414269    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="993ac292-09fd-4434-bce3-e48eaef02a9b" containerName="main"
Jul 07 11:57:37.414389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414279    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerName="istio-proxy"
Jul 07 11:57:37.414389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414285    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerName="istio-proxy"
Jul 07 11:57:37.414389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414294    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a12624-1229-426a-8b93-08fa98d4be01" containerName="maas-api"
Jul 07 11:57:37.414389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414300    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a12624-1229-426a-8b93-08fa98d4be01" containerName="maas-api"
Jul 07 11:57:37.414389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414313    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edd64f6d-0c19-48a6-8d83-3743c0efc734" containerName="maas-api"
Jul 07 11:57:37.414389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414318    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="edd64f6d-0c19-48a6-8d83-3743c0efc734" containerName="maas-api"
Jul 07 11:57:37.414389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414333    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ee46b75-b4f9-46c8-b068-b7a290caf68e" containerName="main"
Jul 07 11:57:37.414389 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414338    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ee46b75-b4f9-46c8-b068-b7a290caf68e" containerName="main"
Jul 07 11:57:37.414768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414396    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="993ac292-09fd-4434-bce3-e48eaef02a9b" containerName="main"
Jul 07 11:57:37.414768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414404    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy"
Jul 07 11:57:37.414768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414409    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a12624-1229-426a-8b93-08fa98d4be01" containerName="maas-api"
Jul 07 11:57:37.414768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414414    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerName="istio-proxy"
Jul 07 11:57:37.414768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414420    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="edd64f6d-0c19-48a6-8d83-3743c0efc734" containerName="maas-api"
Jul 07 11:57:37.414768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414426    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f3cabc-017f-4773-be7e-02844a5c9dbb" containerName="istio-proxy"
Jul 07 11:57:37.414768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414433    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6c61125-4862-4011-99f0-cf8307f6e4a6" containerName="istio-proxy"
Jul 07 11:57:37.414768 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.414439    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ee46b75-b4f9-46c8-b068-b7a290caf68e" containerName="main"
Jul 07 11:57:37.417664 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.417647    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.421368 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.421344    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k5px7\"/\"default-dockercfg-7s2dd\""
Jul 07 11:57:37.421483 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.421374    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5px7\"/\"kube-root-ca.crt\""
Jul 07 11:57:37.421483 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.421390    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5px7\"/\"openshift-service-ca.crt\""
Jul 07 11:57:37.425283 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.425262    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"]
Jul 07 11:57:37.532412 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.532381    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-podres\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.532560 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.532423    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-lib-modules\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.532560 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.532458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-sys\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.532560 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.532490    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-proc\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.532560 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.532513    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9pvn\" (UniqueName: \"kubernetes.io/projected/0588f499-e808-4ac1-9064-3fddf51e7514-kube-api-access-n9pvn\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.633267 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.633233    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-lib-modules\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.633391 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.633298    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-sys\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.633391 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.633349    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-proc\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.633391 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.633380    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9pvn\" (UniqueName: \"kubernetes.io/projected/0588f499-e808-4ac1-9064-3fddf51e7514-kube-api-access-n9pvn\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.633501 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.633423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-podres\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.633501 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.633437    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-lib-modules\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.633501 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.633443    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-proc\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.633501 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.633435    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-sys\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.633633 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.633533    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0588f499-e808-4ac1-9064-3fddf51e7514-podres\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.641049 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.641021    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9pvn\" (UniqueName: \"kubernetes.io/projected/0588f499-e808-4ac1-9064-3fddf51e7514-kube-api-access-n9pvn\") pod \"perf-node-gather-daemonset-526s9\" (UID: \"0588f499-e808-4ac1-9064-3fddf51e7514\") " pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.710874 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.710794    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/1.log"
Jul 07 11:57:37.719160 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.719133    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nbjlj_228f0fc4-cb8d-4540-97cb-3f794bbbeb63/console-operator/2.log"
Jul 07 11:57:37.728366 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.728348    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:37.865475 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:37.865446    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"]
Jul 07 11:57:37.868850 ip-10-0-133-180 kubenswrapper[2582]: W0707 11:57:37.868827    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0588f499_e808_4ac1_9064_3fddf51e7514.slice/crio-4731b17c1b90fd3703f6b997f881d4259b5c2104543a3499fb3de079f0cf6e11 WatchSource:0}: Error finding container 4731b17c1b90fd3703f6b997f881d4259b5c2104543a3499fb3de079f0cf6e11: Status 404 returned error can't find the container with id 4731b17c1b90fd3703f6b997f881d4259b5c2104543a3499fb3de079f0cf6e11
Jul 07 11:57:38.854747 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:38.854719    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-75bnk_192da092-b9e0-4ca8-9f23-c752af83ca83/volume-data-source-validator/0.log"
Jul 07 11:57:38.866968 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:38.866943    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9" event={"ID":"0588f499-e808-4ac1-9064-3fddf51e7514","Type":"ContainerStarted","Data":"17c2fa4936ab8bf200c9f744332787b2d2b2c5db14b2863dcb2b16e0270ae509"}
Jul 07 11:57:38.867108 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:38.866985    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9" event={"ID":"0588f499-e808-4ac1-9064-3fddf51e7514","Type":"ContainerStarted","Data":"4731b17c1b90fd3703f6b997f881d4259b5c2104543a3499fb3de079f0cf6e11"}
Jul 07 11:57:38.867108 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:38.867029    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:38.884629 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:38.884587    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9" podStartSLOduration=1.8845742049999998 podStartE2EDuration="1.884574205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:57:38.883060848 +0000 UTC m=+2540.421752261" watchObservedRunningTime="2026-07-07 11:57:38.884574205 +0000 UTC m=+2540.423265595"
Jul 07 11:57:39.978902 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:39.978867    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wxjtb_cc5862d5-881b-445b-9abf-93f146315de8/dns/0.log"
Jul 07 11:57:40.000999 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:40.000966    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wxjtb_cc5862d5-881b-445b-9abf-93f146315de8/kube-rbac-proxy/0.log"
Jul 07 11:57:40.050543 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:40.050518    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ksbdk_360ce88e-8a7a-4e3a-bd30-1d2854664c35/dns-node-resolver/0.log"
Jul 07 11:57:40.648097 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:40.648065    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-57cf9_1113c816-17c6-479c-bb9e-36ea9b537bdb/node-ca/0.log"
Jul 07 11:57:42.080374 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:42.080326    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c5dbcc7d5-crvzt_1de016bd-1463-4d64-ad1b-09e17dce5d18/router/0.log"
Jul 07 11:57:42.761761 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:42.761736    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kmx7n_d99c2e1c-47fd-42ad-ab43-6793dc71a73a/serve-healthcheck-canary/0.log"
Jul 07 11:57:43.257108 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:43.257078    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-rztt6_c98ff6c7-186a-42af-af1e-ba05d65dba1a/insights-operator/0.log"
Jul 07 11:57:43.257765 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:43.257751    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-rztt6_c98ff6c7-186a-42af-af1e-ba05d65dba1a/insights-operator/1.log"
Jul 07 11:57:43.337864 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:43.337844    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cxw5n_157703d9-a98d-4586-999e-ef5f7ff78c97/kube-rbac-proxy/0.log"
Jul 07 11:57:43.356727 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:43.356706    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cxw5n_157703d9-a98d-4586-999e-ef5f7ff78c97/exporter/0.log"
Jul 07 11:57:43.376235 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:43.376189    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cxw5n_157703d9-a98d-4586-999e-ef5f7ff78c97/extractor/0.log"
Jul 07 11:57:44.881916 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:44.881888    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-k5px7/perf-node-gather-daemonset-526s9"
Jul 07 11:57:45.895581 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:45.895546    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723730-g5qgj_2bc5f507-00fe-4de9-8c24-f59641cdf494/cleanup/0.log"
Jul 07 11:57:45.921341 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:45.921301    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723745-lclc5_0df681bd-2339-4c56-92a0-7d29c68724fd/cleanup/0.log"
Jul 07 11:57:46.243356 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:46.243328    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-4kn54_5b2d0fdb-d643-45d2-99bd-851a456b14fa/manager/0.log"
Jul 07 11:57:46.261493 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:46.261474    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59ddd9ddd8-4kn54_5b2d0fdb-d643-45d2-99bd-851a456b14fa/copy-manifests/0.log"
Jul 07 11:57:48.050663 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:48.050634    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65f8ccd9c5-b7jq2_27031715-9c2d-4071-8e38-f3b1af1c6775/manager/0.log"
Jul 07 11:57:48.330836 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:48.330756    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c5dbcc7d5-crvzt_1de016bd-1463-4d64-ad1b-09e17dce5d18/router/0.log"
Jul 07 11:57:49.992299 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:49.992266    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c5dbcc7d5-crvzt_1de016bd-1463-4d64-ad1b-09e17dce5d18/router/0.log"
Jul 07 11:57:52.970600 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:52.970570    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-6ptxs_aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed/migrator/0.log"
Jul 07 11:57:52.989867 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:52.989836    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-6ptxs_aba4bddb-cae3-4bf7-a7f3-84a8cd2e37ed/graceful-termination/0.log"
Jul 07 11:57:53.347513 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:53.347413    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-cwqjb_e8209861-6fc3-4980-8389-3cc58b1d3c40/kube-storage-version-migrator-operator/1.log"
Jul 07 11:57:53.349251 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:53.349224    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-cwqjb_e8209861-6fc3-4980-8389-3cc58b1d3c40/kube-storage-version-migrator-operator/0.log"
Jul 07 11:57:54.408558 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:54.408528    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m464b_0266f5e8-632f-4c24-a508-abaa012b9483/kube-multus-additional-cni-plugins/0.log"
Jul 07 11:57:54.430844 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:54.430823    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m464b_0266f5e8-632f-4c24-a508-abaa012b9483/egress-router-binary-copy/0.log"
Jul 07 11:57:54.450610 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:54.450591    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m464b_0266f5e8-632f-4c24-a508-abaa012b9483/cni-plugins/0.log"
Jul 07 11:57:54.471886 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:54.471855    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m464b_0266f5e8-632f-4c24-a508-abaa012b9483/bond-cni-plugin/0.log"
Jul 07 11:57:54.492078 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:54.492050    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m464b_0266f5e8-632f-4c24-a508-abaa012b9483/routeoverride-cni/0.log"
Jul 07 11:57:54.512903 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:54.512883    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m464b_0266f5e8-632f-4c24-a508-abaa012b9483/whereabouts-cni-bincopy/0.log"
Jul 07 11:57:54.531475 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:54.531455    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m464b_0266f5e8-632f-4c24-a508-abaa012b9483/whereabouts-cni/0.log"
Jul 07 11:57:54.876661 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:54.876639    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-k2fgc_8ba044ce-9149-4c30-b2ac-2317d0bf953b/kube-multus/0.log"
Jul 07 11:57:54.953397 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:54.953373    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rfz9v_0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca/network-metrics-daemon/0.log"
Jul 07 11:57:54.969542 ip-10-0-133-180 kubenswrapper[2582]: I0707 11:57:54.969512    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rfz9v_0a63e931-4e3a-4e6d-9a17-b0a6dd83ccca/kube-rbac-proxy/0.log"